I just noticed the deadline for the final report was extended to June 17 @ 5 pm PT. I found out last night by looking at the front page of the competition; there was no announcement in the forums and we did not receive an email. It seemed highly unusual that a deadline was extended a week after it had passed, with no announcement or explanation. Can the organizers better communicate why this change was made, and when we can expect the final validation results to be posted?
Created by PAUL PERRY paulperry In order to achieve the computational goals of the Metadata Dream Challenge, we understood that we needed to encourage as much participation as possible. Given that our challenge occurred during the height the coronavirus pandemic with rounds starting in early Feb ? May, we realized that many participants may have been impacted by shifting to working from home, shuttered businesses, layoffs and lost jobs, lost family members, or becoming sick themselves.
So, on May 27th when we noticed almost 50% fewer teams submitting to the Validation Phase than submitted in Round 3, we felt an extension from May 29 to June 2 was warranted to encourage as many submissions as possible. The extended date resulted in a few additional submissions, none of who were in the final top 3 performers, and in the end, the top 3 performers also submitted their write-ups on time.
We apologize when on occasion we didn't communicate as quickly as we should have, as mentioned by @fragosog. It was a learning lesson for us as well. We thank you all for your patience and participating in this Challenge.
Hi @ehartley
> I also didn't receive the notice about the deadline extension and neither did any of my teammates.
Ah, that's not good. I apologize sincerely, personally as well as on behalf of the organizing group.
> disappointed to learn that our efforts to get the write-up submitted on time on June 5th were wasted nd that the other teams . . . have now had an extra week to work on it.
I understand. We failed in that this notice should have gone out sooner so that everybody benefitted from the extra time, or not at all.
Regards,
Gilberto I also didn't receive the notice about the deadline extension and neither did any of my teammates. We found out about it Monday night (6/15) when I checked the Leaderboard to see if the results had been posted yet. We were disappointed to learn that our efforts to get the write-up submitted on time on June 5th were wasted and that the other teams (though not all of them based on @paulperry's post) have now had an extra week to work on it. Hello @paulperry
Apologies about this. I can't say what happened with the notice but it went out last Friday, I received it around 6 PM, but maybe it didn't get to the other participants either so I'll reproduce it below (although it might not be helpful at this point if others didn't receive it). Yes, there were three groups that hadn't submitted by the deadline, so to compensate for any delays related to people working on covid or affected by the pandemic we decided to extend the submission deadline to give the teams extra time. A few submissions could certainly have incluced more details in the writeups. [We later found that two submissions had been posted to the wrong location.] We are meeting this week to review the writeups and the validation phase results. I expect that the results will be posted soon after this.
Regards,
Gilberto
Dear Metadata Automation DREAM Challenge Participants,
On behalf of the National Cancer Institute, Sage Bionetworks, Clinical Data To Health (CD2H), and the DREAM Challenges organization, we would like to thank you for your submissions to the validation phase! As we assess the final scores please submit the following below if you have not already done so.
The deadline for the writeup submission has been EXTENDED to Wednesday June 17th at 5PM PT. Write-ups must still be submitted in order to list as an consortium author on the challenge paper. In addition, your source code and dockerfile must also be submitted to be eligible for incentives. For more information on submitting your write-ups please visit the Writeup Submission section on the challenge site.
Please continue to post your questions and any issues you are having with the Challenge on the discussion forum. We thank you for your submissions and efforts towards this challenge!
Have a great weekend! Stay safe and be well!
Regards,
Metadata Automation DREAM Challenge Organizers