I ran a submission in Sub Challenge 2 main queue yesterday that failed in such a way that no useful prediction data was produced. Unfortunately, the script ran to completion and produced an output file. The output file would not have contained any valid predictions. Would it be possible to get the submission cancelled so that I can try again with the issue resolved?
You can verify that the submission didn't produce any meaningful predictions: Submission name: lb2_ch2_jc Submission ID: 9639862
Thanks,
Created by Jason L Causey jcabio Thank you, I'll be more careful to check outputs and exit the script early in case of unexpected output in the future. That submission is now invalid, so you can resubmit. Dear Jason,
Your prediction file did produce properly formatted columns but had identical continuous prediction scores within each cohort. I will review this with some internal challenge organizers and most like will invalidate this submission for your team. However, please be aware that in the next leaderboard round and in the final validation round we will not be accommodating teams in this manner given that these rounds have financial awards associated with them.
Kind Regards,
Mike Since the organizers have done so much work to sanity check the submission output and reject ones containing errors, I think it's agreeable that buggy submissions should not be counted into the quota.