As is shown on 2.3 section of the challenge wiki, there are five validation datasets. Will these validation datasets be fed into a submitted model and their AUC evaluated separately? Or will the AUC of the merged validation dataset be evaluated?
Thanks a lot!
Created by hongjie chen chenhongjie Dear Aditya,
The problem is now resolved. I apologize for any inconvenience. @Michael.Mason
The validation files named sc*_SimmulatedValidation_ClinAnnotations.csv appear to be empty (I only see three 4-byte files containing "x", as opposed to what's written in the descriptions about these validation files). Could you re-upload them?
Thanks,
Aditya Hi Hongjie,
There are simulated validation clinical files [here](syn7222257). With names of studies in them. They are some smaller random samples of the full sample set but they have the study names in them.
Kind Regards,
Mike Hi, Mike!
Do we know which subset of validation data sets is used for some sub-challenge?
Thank you! That helps! Thank you very much! Hi,
Please review the two links below:
[here](https://www.synapse.org/#!Synapse:syn6187098/wiki/449443) and [here](https://www.synapse.org/#!Synapse:syn7222257)
While there are 5 validation sets, each challenge questions will use as subset of 2-3 appropriate cohorts in the validation. Your code can will point to them and read them as you like you can predict as you see fit: all at once, each validation cohort at once, or each sample individually. How you normalize/map data will affect this. For example in Challenge question 2, a predictor could have two separate approaches to microarray data and RNA-seq or it could work to map them to the same space and predict. How you do it is is up to your team.
good luck,