Hello ${badge?isUser=false&id=3397771},
### Validation Phase has Started!
The validation phase has started! This means that you have until March 16 5PM PST to submit your final models to the validation queue for competition. The individual or team with the highest score will win! See [this thread](https://www.synapse.org/#!Synapse:syn20940518/discussion/threadId=6803) for more specifics about the submission queues, and [the wiki](https://www.synapse.org/#!Synapse:syn20940518/wiki/600161).
**Note** one gotcha from the validation dataset that your submissions must handle is that some inhibitors were not tested on any drugs in the validation dataset. You can see how beataml_example2 needed to be modified in [this commit](https://github.com/Jacoberts/beataml_example2/commit/da698263cc1595ef4ee430bdfe0051902d878449).
### Secondary Metrics
We have implemented secondary metrics for both subchallenges. This rule change was done to match a DREAM challenge convention. It will likely not effect your training as it is only used in the event of a relative tie. For SC1 we are using the Pearson correlation as the secondary metric, and for SC2 we are using the [cumulative/dynamic AUC for right-censored survival data](https://scikit-survival.readthedocs.io/en/latest/generated/sksurv.metrics.cumulative_dynamic_auc.html) evaluated at one year. [The wiki](https://www.synapse.org/#!Synapse:syn20940518/wiki/600161) has been updated. Again, the secondary metric is only used as a tie-breaker.
Please continue to use the discussion board for any questions!
Best,
Jacob from the CTD^2^ BeatAML challenge team
@v.chung
Thanks a lot Verena for your reply. I have followed the steps laid out by you and hopefully can now receive direct messages from any Synapse user.
Also, thanks for informing about the submission going through !!
Regards,
Team Resham @raghvendramall,
Were you able to successfully change the team settings when you followed the steps [here](https://www.synapse.org/#!Synapse:syn20940518/discussion/threadId=6659&replyId=21282)?
And my apologies, I looked into your submission further and it went through successfully.
Sorry about that!
Verena Hi @v.chung
Thanks a lot for your reply. What would be a fix to allow email notifications from any Synapse users.
As per your second comment that the model gave invalid results, its very surprising as I just submitted the same model in the leaderboard phase and it performs perfectly well. Since, we cannot see the logs in the validation phase any help regarding this would be crucial.
Thanks a lot in advance.
Regards,
Raghvendra Hi @raghvendramall,
It looks like your team is still not allowing for email notifications from any Synapse users. However, I can let you know here that your earlier submission was invalid. And yes, you are allowed to continue submitting until there has been a successful (e.g. scored) submission.
Best,
Verena HI @Jacoberts ,
I made my one submission for the day for the validation phase about an hour ago. I have not yet received any notification about whether the model succeeded or failed in this phase.
Did the validation phase start only after you posted your comment?
Also, for the one submission per day, will I be allowed to submit till one model succeeds during that day?
Regards,
Team Resham
Drop files to upload
Validation Phase has started, and new secondary metrics page is loading…