Hello ${badge?isUser=false&id=3397771},
### Leaderboard Round (ends March 2)
The Validation round starts in one week! What this means for you is that **on March 2 at 5 PM PST**:
- the leaderboard will freeze
- the leaderboard dataset will be released under the "Files" tab
- the validation queue will open up as a new option under "Submit Docker Repository To Challenge"
### Validation Round (ends March 16)
The validation round will last until March 16, 5PM PST, when the validation queue closes. **During the validation round**:
- you can submit at most 1 valid submission per day to the validation queue
- validation submissions will report success or failure, but you cannot see your score or your logs
- the leaderboard queue will remain open for testing purposes, but scores won't go on the leaderboard
Your score from your final submission during the validation round. If you have any issues during these final three weeks, don't hesitate to post in the discussion forum.
Happy modeling!
Jacob from the CTD^2^ BeatAML challenge team
Created by Jacob Roberts Jacoberts Dear @ken89,
Apologies! It looks like the predictions file created by your model has slipped through our validation checks, in that there are missing AUC values in the third column (hence those NaN scores). In short, the submission is considered invalid.
I will put in a fix now, so that these submissions are not counted against your submission limit.
Best,
Verena Hi!
My team has made a successful submission but your email states that:
"Your submission has been scored. Results will be announced at a later time."
"Your submission (syn 21679510) is scored, below are your results:
spearman: nan
pearson:nan "
Does this mean my team's submission is successful? Dear @ElsaB,
Thank you for participating!
You are allowed one valid submission per day from now until the phase end on Monday, March 16th, at 5PM Pacific Time. That is, you may submit to each validation queue once a day and every submission will be scored. Note, however, that we will only use the the latest submission as the final submission. More clarification about the validation phase can be found in this [thread](https://www.synapse.org/#!Synapse:syn20940518/discussion/threadId=6799&replyId=21553).
Let us know if you have any additional questions!
Best,
Verena Dear Jacob,
Thank you for the explanation.
Quick question/clarification:
Are we only allowed 1 validation submission or several?
ie only our last validation submission will be scored or all of our validation submissions will be scored?
Thank you much
All best
Elsa
Dear Team Resham,
Thanks for clarifying!
a) Yes! You are encouraged to use (training + leaderboard) data to train your models.
b) Yep. Of course, if you're using the leaderboard dataset to train, your score will be artificially high.
Best,
Jacob Dear Jacob,
Thanks a lot for this detailed information. I have some queries and would be grateful if you can clarify the same.
a) When the leaderboard dataset is released, can it be used to make new models?
b) If we make new models and submit them to the leaderboard, can we get their scores? (I think it is answered by your 3rd bullet point in the Validation Round information above but just want to double-check)
With Regards,
Team Resham
Drop files to upload
One Week Left in Leaderboard Round page is loading…