Hello, Apologies if this is somewhere else, but from what I've seen, it looks like the max wall time for inference submissions in the validation phase is 18 days, compared to 12 for the leaderboard phase. Should we then assume that there is 50% more data in the validation phase inference, i.e. if our submission took 12 days to run previously, now we'd expect 18? Just trying to estimate compared to previous submissions. Thanks! Bill

Created by Bill Lotter bill_lotter
Thanks Thomas!
Hi Yaroslav, > did you gather all the data from all the hospitals and then spitted into trainining / testing for the 3 Leaderboard rounds / testing for the Final Round sets? Yes. We split the data so that the population of each split has the same distribution as the global dataset. Thomas
Hello @tschaffter One more question on the same topic: how did you generate training-test split? I mean, did you gather all the data from all the hospitals and then splitted into trainining / testing for the 3 Leaderboard rounds / testing for the Final Round sets? I ask because I'd like to understand if the data comes from exactly the same distribution. Because if the testing set in Final Round comes from a separated hospital whose data was never seen before, it can also cause some normalization issues (due to a different device, acquisition issues, etc) and we should probably re-think the normalization procedure and regularize further the model. Again, in short, does the testing set in this Final Round come from exactly the same distribution as the current training set? Thank you, Yaroslav
Okay sounds good thanks Thomas!
Hi Bill, > Should we then assume that there is 50% more data in the validation phase inference. Yes

Inference Data in Validation Phase page is loading…