Dear Participants,
Thank you to those who submitted the work for the challenge.
We have to add several notes on the submission status as we received several questions regarding this through e-mail.
You can check the status of your submission on your dashboard. We used 3 statuses in this challenge.
A. **INVALID**: if we cannot evaluate your submissions. The reasons varied but included: 1) we cannot find the command to run your submissions (for those uploading .zip file), 2) Your submissions can be run, but generate error which we will email you with the log so you can correct your submission accordingly, 3) We cannot find the requested output file, scores.csv, so the statistic of your submissions cannot be updated. Please follow the submission guidelines. Failing to do so may also result in INVALID submission. We may **stop** processing your submissions if you submit more than 3 INVALID submissions, thus please check that your submitted workflow work in your local system without error. The error might only appear in our system such as discussed in this [thread](https://www.synapse.org/#!Synapse:syn27130803/discussion/threadId=9858], so it is always good idea to check with the forum to make sure that the error is not coming from our system. Otherwise, we will consider your submission as INVALID.
B. **CLOSED**:if we consider your submission as a duplicated submission based on synapse ID. Please make sure that the new submissions are different synapse entities or have different synapse ID. Although the contents might be edited, our workflow will consider it as duplicate based on synapse ID. We implemented this because we would not process the same synapse entity twice to make sure efficiencies and we are not evaluating the same submission twice. We are very sorry for the inconvenience. This status will not be notified through e-mail, please make sure to check your dashboard. Submissions might also could appear as duplicated if you use the same filename with your previous submission, so please use the new name. Please see the notes below on how we will count your VALID submission. We will not use the filename to count your submission, so it is always good idea to update the file name, once the old file name has been evaluated (i.e the status has been changed from RECEIVED to other status, for example, SCORED).
C. **SCORED**: if your submissions pass through our workflow smoothly and the statistic (Harrel's C and Hosmer Lemeshow) can be tested. We will e-mail you with the status and score once your submissions is SCORED. The score also available in the [Leaderboard](https://www.synapse.org/#!Synapse:syn27130803/wiki/619288). Please notes that the rank is generated by the main metric Harrell's C score.
We will limit your submission in SUBMISSION PHASE up to **5 submissions**. This will be counted based on **SCORED** submission, however, once again, if **we received 3 INVALID submissions in a row** from the same team, we might not consider your next submission for evaluation.
If you have general questions, please discuss through discussion forum so others can be benefited by the discussions. Otherwise, if you have questions related specifically to your submission, you can contact us through e-mail.
Any generic questions may not be replied by e-mail, so please be sure to use the discussion forum.
Best regards,
FINRISK DREAM Challenge Organizers