Dear Participants,
We are excited to let you know that starting today (July 8th), you can make your first set of submissions to the challenge that will be scored on a leaderboard at the end of the week. You have an opportunity to make 1 submission each week until July 28th and receive a rank on the leaderboard at the end of each week. The final submissions are due August 1st.
Submission Due Dates:
Round 1: July 14
Round 2: July 21
Round 3: July 28
Final submission: August 1
Let us know via the discussion forum if you have any questions.
Good luck to all,
The Malaria DREAM challenge Organizers
Created by Geoffrey Siwo gsiwo Dear Participants
Round 2 submissions are due July 21 midnight PDT.
Thank you
Geoffrey Dear Organisers,
Round 2 is open now?
Best,
Jinseub Dear all,
Please note that we are not providing the actual evaluation scores on the leaderboard. This is to avoid potential overfitting on the leaderboard especially given the small sample size of the test set. Scores will be provided only on the final submissions.
Thanks,
Geoffrey Dear all,
Apologies, but there has been an update to the leaderboard as it was missing one submission due to a technical error.
@RNI_group, I will let @gsiwo respond to you about the correlation return.
Best,
Tom
I forward emails.
Thanks Dear @jinhwa,
Will you forward me the scored email you got at thomas.yu@synapse.org?
All your submissions I can see are invalid.
Best,
Tom Our team name is MD_DUTeam.
Thank you for your effort.
Best
Jinseub Thank you for opening up the leaderboard. I think perhaps the correlations that should be posted there and not just the relative ranks. Dear all,
The leaderboard is found here: https://www.synapse.org/#!Synapse:syn16924919/wiki/594001.
@jinhwa, what is your team name that you submitted as?
Best,
Tom Apologies, but we didn't receive any score so far. Also I am unable to locate the leaderboard for this challenge. Dear Organisers
There is no our team information in the leadeboard for subchallenge 1.
We received an e-mail that our submission has been scored.
Does the leaderboard only show up to rank 16 ? Or better prediction than random?
Best,
Jinseub Dear all,
The leaderboard will be posted this morning.
Best,
Tom When and how would we get feedback on Round-1 submission? So far there's no leaderboard, hence we were wondering. Dear Bence,
Round 1 submission deadline is Sunday July 14th midnight PDT. All the other deadlines below will also be midnight PDT.
Round 1: July 14
Round 2: July 21
Round 3: July 28
Final submission: August 1
Best wishes
Geoffrey Dear Organisers,
what time is the end of submission deadline? Sunday midnight PT?
Thanks,
Bence Dear all,
The write-ups should be submitted for the final submission which will be another queue. The leaderboard will become available on the morning of 15th PDT.
Best,
Tom Can we see the results of our submissions somewhere or do we just wait until Leaderboard becomes available after July 14th? Dear Nicola,
Thank you for your question. For the leaderboard, only the predictions are necessary. But if you are able to, you can also submit code and a write-up which would be beneficial for documenting your work.
Geoffrey Dear Bence,
Thank you for the suggestion. Yes, we will provide a baseline prediction as a reference model.
Geoffrey Thanks for this opportunity to check our work. Should we also post code and a write-up for these leaderboard submissions, or are only our outputs necessary?
Cheers
Nicola Dear Geoffrey and Organisers,
I think this kind of leaderboard is a great idea, as gives us some feedback without the danger of overfit. One idea: maybe you (the organisers) could also submit a "baseline" prediction, the results of some very basic model you made based on the training data. This would also help us to have some insight whether our predictions perform "reasonable" well - not only compared to each other, but also compared to some reference model. I think this could be help a lot in especially in case of SC2, where we have very different training and test sets. Hopefully comparing our models to this reference prediction also won't lead to overfit. Thanks,
Bence