Dear contest organizers, Would it be possible to provide examples of downloading deep net models trained via Docker? Would it also be possible to release example testing scripts to accompany the sample training and preprocessing scripts that you've released? (Apologies if these examples already exist.) Will leaderboard / test set benchmarks also be released? Am I correct that the leaderboard, test, and training sets all come from the same population of patients, and that those patients were randomly assigned to one of the three datasets? Thanks!

Created by DREAMer
> Is it also correct that the leaderboard, test, and training sets all come from the same population of patients Yes, we are using the data that we have received from Group Health. > , and that those patients were randomly assigned to one of the three datasets? We have split the subjects from Group Health so that each of the three sets are representative of the global population. > Are there instructions for downloading the model after a training submission has stopped running? A link to download the trained model is included in the dashboard, however the feature is not enabled yet. It will be activated on the Express lane sometime before the beginning of the competitive phase.
Thanks Thomas! I have two additional questions: Is it also correct that the leaderboard, test, and training sets all come from the same population of patients, and that those patients were randomly assigned to one of the three datasets? Are there instructions for downloading the model after a training submission has stopped running?
> I'm wondering whether you will report the performance of baseline methods on the leaderboard and test sets. This is still something that we are discussing for the Leaderboard Phase, but in principle yes starting from the Second or Third Round. Regarding the Validation Phase, participants will have only one shot to evaluate their method against the test set. The results will be released at once some time after the completion of the Competitive Phase.
> Do you mean instructions on how to download the model after a training submission has stopped running? Yes, exactly. > however we can release an example of testing script that generate a dummy prediction file. Great, even a dummy file would be helpful. > The leaderboard and test sets, as well as the training set, are not available for download. I believe my original question was unclear. I'm wondering whether you will report the performance of baseline methods on the leaderboard and test sets.
Hi DREAMer, > Would it be possible to provide examples of downloading deep net models trained via Docker? Do you mean instructions on how to download the model after a training submission has stopped running? > Would it also be possible to release example testing scripts to accompany the sample training Assuming that a method is trained to make prediction at the image level, some thinking as to be done to generate a prediction at the breast level. We are not going to make suggestion on how to do that, however we can release an example of testing script that generate a dummy prediction file. > Will leaderboard / test set benchmarks also be released? The leaderboard and test sets, as well as the training set, are not available for download.

Examples of testing and instructions for downloading trained models page is loading…