Dear organizers,
Congratulations for all the hard work done so far! I was wondering if you have any estimation of the final date for the challenge. Christmas and new year holidays are comming and it would be good to have an idea to get the schedule organized. Thank you very much for your efforts!
Best
Martin
Created by Martin Guerrero martinguerrero89 Hi Brian!
Is good to know you are doing well!
I'm excited to hear the news, I find the schedule very convinient but I'm also open to any changes due to the unusual circumstances.
Thank you for your time and response
Best
Martin Dear @martinguerrero89 and all,
I hope you are all well. I sincerely apologize for the delays in this Challenge. We have now generated and QC'ed the validation data to our satisfaction.
Despite the uncertainty and worse that the coronavirus has introduced to our lives, we will launch the validation phase on April 1st. We want to be as flexible as possible to accommodate our shifting realities during this time. As such, we will leave the validation phase open for _at least_ a month, closing not before April 30th 5PM Pacific US time. Please let me know publicly or via brian.white@sagebase.org if this creates difficulties for you. We will gauge participation in the coming weeks and your responses in order to see whether we need to extend the validation phase further.
Here are the current plans:
1. Validation phase opens April 1st 9AM Pacific US. Closes April 30th 5PM Pacific US.
2. Limit of two submissions per team.
3. Primary scoring metric will be the mean-of-means Pearson correlation--i.e., the mean Pearson correlation across cell types within a dataset, averaged over datasets. The secondary metric will be the mean-of-means Spearman correlation.
4. You will receive the following "scores" for each submission: For each sub-challenge, the mean-of-means Pearson and Spearman correlations, the per-cell type and per-data set Pearson and Spearman correlations, and the per-cell type (averaged over datasets) Pearson and Spearman correlations.
5. The leaderboard data will be updated in "near real-time" (likely every hour) and will include the scores from (4).
Please let us know how we can facilitate your continued participation.
Best,
Brian
Hi!
I hope everyone is doing well and safe during these hard days
Any update about the status of the challenge?
Best,
Martin Dear organisers,
To echo Martin's post, do you have any updates regarding the timeline of the challenge? At least some of us would find it useful for organising their project schedules.
Thank you for keeping the challenge going despite the difficulties with the validation data!
Best,
Miro Hi Brian,
any news about the validation data? Any update would be well received.
Best!
Martin Hi Martin,
We are having some difficulty with the validation data that is pushing our timeline back. I suspect we won't be able to launch the validation phase until late January. I'm sorry for the delay. I will make a separate forum post.
Brian