Dear Participants, Thank you for your enthusiasm for FINRISK - Heart Failure and Microbiome DREAM challenge. We are excited to announce that we are ready to resume the evaluation process. Thank you for those who already submitted the work before December 8th, we have evaluated your submission. Please check your dashboard and/or separated e-mail. We are sorry again for the delay. We gather several notes from the received submission. Please check all the points below to make sure that we can process your submission smoothly. We are sorry for not making this points clear before. 1. Please make sure to follow the guideline provided in [submission tutorial](https://www.synapse.org/#!Synapse:syn27130803/wiki/619286). 2. Make sure to name your submission file as: **Team_Name_Submission_Number**. This will allow us to easily track your submission in the event we need manual check-up. Therefore please avoid generic name such as **submissions_1** or **FINRISK_1**. 3. Please provide 1 argument as the input to your submission so that we can point to evaluations directory which contains real dataset. The folder structure is similar to those of DreamHF.zip, so the test and training folder is nested inside the upper directory (the name is not DreamHF, but the content is similar with example). 4. Upon downloading your submission, we, organizers, will create a list that linked your submission ID and your submissions file name. Afterward, we create the directory name as **your submissions file name** containing your submitted file. Inside this directory, we also will create a folder named `output` to place your `scores.csv`results and statistic of the score (harell's C and hosmer-lemeshow P value) and `log`to record any output and error from your scripts. Please make sure to set your script to follow this informations. Failed to do so will results in *INVALID* submission, as our workflow cannot find your file to score. 5. Please make sure to make a right file name. For example, the output of your algorithm shoud be `scores.csv`, other name can result in **INVALID** submission because our workflow cannot find your file. 6. Please follow the gold standard of `scores.csv` [here](https://www.synapse.org/#!Synapse:syn48276490). This file should include 2 columns name "SampleID" and "Score". The "Score" should be ranged in 0-1,with larger numbers being associated with higher probability of having HF. The values, -Inf, Inf and NA, are not allowed. Fail to do so can result in **INVALID** submission because our workflow cannot update your score, for example because it failed to find the columns name requested. The baseline model example has been provided [here](https://www.synapse.org/#!Synapse:syn49071631), please see the README.md for more information. 7. Importantly, as we updated [here](https://www.synapse.org/#!Synapse:syn27130803/discussion/threadId=9811), for those who submitted .zip file, please provide us with 1 file, name *Run*, that contain 1 line command to execute your submission. Failed to do so can results in *INVALID* submission as we relay on our workflow to find the command. This especially helpful if you have complex nested folder structure inside your *.zip submission. Finaly, we provide you with script on how we calculate the harrel's c index for model accuracy and hosmer lemeshow test for model calibartion on your submission [here](https://www.synapse.org/#!Synapse:syn48276655). We updated recently that we will test whether the model you provided is well-callibrated for 15 years (not 10 years, please see update [here](https://www.synapse.org/#!Synapse:syn27130803/wiki/619280)). Thank you again for your understanding and we are excited to evaluate your submission. Don't miss the deadline, the timeline is [here](https://www.synapse.org/#!Synapse:syn27130803/wiki/616705). Thank you and please don't hestitate to contact us through discussion forum if you have any further questions. Best regards, FINRISK DREAM Challenge Organizers