Dear Organizers, I would like to ask several questions about the last round that would probably help me and all of us to optimize our performance. - In another thread I read that we have just one submission per Sub-Challenge. What if this final submission is worse than a submission from one of the previous rounds, which one will be used for the final leaderboard? - Will we have the same training and testing data or will you use unseen test set, for instance? - Do you limit somehow the inference time? It is pretty easy to create a submission which would run for weeks. I think it is good for all of us to set a reasonable limit on inference time (I don't know, 2-3 days?). Thank you Yaroslav

Created by Yaroslav Nikulin (Therapixel) ynikulin
Hi Yaroslav, > What if this final submission is worse than a submission from one of the previous rounds, which one will be used for the final leaderboard? All the participants will be evaluated on the same (validation) dataset (there is a link with your second question). > Will we have the same training and testing data or will you use unseen test set, for instance? In the Validation Phase, the training set will be the union of the training and scoring data used during the Leaderboard Phase. The scoring set in the Validation Phase will be composed of unseen data that are representative of the population of the Leaderboard training and scoring sets. > Do you limit somehow the inference time? We have to set a wall time (maximum amount of time that a submission is allowed to run) for submissions otherwise we may still be running submissions in one year. The current value for the wall time for inference submissions in the Validation Phase is 18 days but we will confirm it in the newsletter that will be sent just before the beginning of this final phase. Thanks!

Several questions about Final Round page is loading…