**Email sent to participants on 9/6/2016**
Dear ENCODE-DREAM Challenge Participants,
We would like to inform you of several updates that have been made in the ENCODE-DREAM in vivo transcription factor binding site prediction challenge.
1) The submission queue for the final scoring round is now open!! Note that for each final round TF/cell type combination, you can submit predictions as many times as you like, but the scoring will be done only on the last submission you/your team made on or before the September 30th close of the final scoring round. Teams will get an email notification once they have made a submission that will indicate if the submission is valid or not. But you will not receive any feedback on the performance of any final round submission. Also the "Final round submission status section" (Section 5) on the wiki will be updated with date and valid status of every submission, but no scores.
2) For final submission, please note that when submitting models, please submit them to the ?ENCODE-DREAM Transcription Factor Binding Site Prediction Challenge (FINAL ROUND)? queue, noted in Section 3.4, step 6: https://www.synapse.org/#!Synapse:syn6131484/wiki/402044
3) For final submission, please remember to use the final round submission template which is different from the leaderboard template (in that it includes coordinates across the whole genome). The templates are provided in Section 3.4 on the wiki https://www.synapse.org/#!Synapse:syn6131484/wiki/402044
4) For final round submissions, remember to name your files appropriately (starting with the F prefix). File naming instructions are in Section 3.4 on the wiki https://www.synapse.org/#!Synapse:syn6131484/wiki/402044
5) Note that the TF/cell type combinations that you need to submit for the final round are different from the ones in the leaderboard round. See https://www.synapse.org/#!Synapse:syn6131484/wiki/402034 for the full list.
6) The leaderboard round will remain open throughout the challenge. You can continue submitting to the leaderboard round as you have been; using the appropriate template file, submission queue and file naming convention. Note that the number of submissions remains capped to 10 for each TF/cell type combination in the leaderboard round.
7) This week, we will be updating the leaderboard scoring scripts to incorporate the integrated score that will also be used in the final scoring round. We will also release the code for the integrated scoring over the weekend.
8) Finally, we have decided to extend the Challenge beyond the September 30 deadline to encourage broader engagement in the community. We have gotten feedback that the Challenge is difficult and on a short time frame, thus we will run a second phase of the Challenge that will run until January 7, 2017. The Challenge will proceed as planned until September 30, where final submissions will be scored and the top performing teams will be invited to give a presentation at the Recomb/ISCB Conference. This will end the first phase of the Challenge. Starting on October 1, 2016, we will reset the submission queues for the leaderboard and the final scoring round. You will be allowed another 10 submissions per TF for the leaderboard round and allowed 1 final submission (scored on the last submission made).
If you have further questions or need clarifications, please post your questions to the Discussion Forum on the Challenge website.
We encourage everyone to attend the Recomb/ISCB Conference on Regulatory & Systems Genomics with DREAM Challenges & Cytoscape workshop: https://www.iscb.org/recomb-regsysgen2016
Good luck with your final submissions!!
Kind Regards,
The ENCODE-DREAM Challenge Organizing Team.
Created by James Costello james.costello Hi,
What is the id for the final round submission queue? As far as I understand the id for the leaderboard submissions is 7071644 but the final round has a different queue. I had a look around to see if it was documented anywhere but I couldn't find it. I might have missed it as there is quite a lot of information spread around the wiki and the discussion threads.
Thanks in advance,
Epi. Hi Yichao,
Yes, that is correct. Given the tight timeframe, we will be allowing participants to submit their write-ups after September 30, due on Oct 14.
Cheers,
Jim
so the write-ups can be submitted after September, 30?
Thanks,
Yichao Hi Ivan, zhicheng, and Lawrence,
I hope the email (copied above) answers your questions. Please post here if need any clarifications or have any additional questions. Thanks for being active members of the community.
Cheers,
Jim
**Email sent to participants on 9/7/2016**
Dear ENCODE-DREAM Challenge Participants,
A few questions arose from the previous email that was sent on September 6 that we would like to clarify.
1) We will keep the September 30 deadline for final submissions for this round of the Challenge. We will need to score and then invite teams to the Conference in November and this is a tight timeframe so we will not be extending the September 30 deadline.
2) We will honor our original proposition for byline authorship for the top performing team ending September 30. We will also extend this offer to the winners of the extended round ending January 7. The overview paper that we will be submitting to Cell includes the description of the challenge, the overall results, the top performing methods, and post-challenge analysis, including comparisons to the baseline predictors and evaluation of features, methods, etc.
3) For the September 30 deadline, we will NOT require a public release of code or method write-ups. We will be asking all teams that make a submission to provide a write-up of their method by October 14, 2016, but to kept private.
4) The organizers will contact top performing teams directly after September 30 and work with them to validate their models. Top performing teams will be presenting their method a the DREAM Conference, so we will require a write-up that will be made public.
5) Public release of code, method write-ups and post-challenge benchmarking will be required for the January 7 deadline.
We appreciate the questions and feedback, so please keep asking and we will address them as soon as we can.
For the sake of everyone?s inbox, please post any questions to this thread. We intend to follow up with any updates using this thread.
Thanks again for everyone?s active participation and best of luck for the September 30 deadline.
Kind Regards,
The ENCODE-DREAM Organizers Thanks for the extension!
At the current rate it takes more than 48 hours to train my model on a CUDA GPU enabled PC. I need more time or more compute power! The extension is much needed. For the overall performance (to determine the final winner and probably the authorship of the Cell paper), only the results from the 2nd round phase will count? Dear James,
Thank you for a detailed update, the updated two-stage setup seems very convenient.
Can you provide more information about the method write-ups?
What is the deadline for the first phase write-ups? (September, 30?)
Will the 1st phase write-ups become 'public' to be taken into account by participants of the 2nd phase of the challenge?
Thank you,
Ivan
Drop files to upload
Final Submission Queues Open!!! Additional notes on the Challenge page is loading…