I have successfully submitted scoring in express lane sub-challenge 1. But when I use the same scoring script in regular lane, I hit an error (submission id: 7799685) ``` SubjectIds must be part of the gold standard ``` How do I check if a subjectid is part of the gold standard?

Created by Yohanes Gultom yohanesgultom
Thank you, @thomas.yu, that solved the issue.
Dear @ynikulin After taking a quick peek at your predictions file, it seems that you actually don't use the correct laterality column. Laterality has to be L, and R, but you have MLO and CC as the values. Best, Thomas
Dear @yuanfang.guan, Apologies for the confusion. I recently updated the email that was being sent out to participants, but the scoring code is exactly the same. Please kindly provide me with your submission Id and I will check your prediction file. Best, Thomas
yellow thomas yu, i think you just changed your scoring code, i didn't receive the proportion that i missed (didn't make predictions), while i am absolutely sure i missed a big chunk, can you please check? also, i didn't get this format of report: Your submission was missing predictions for 0.000000% of the samples, the confidence values of these samples will be filled in with the median. Your scores are: auc: 0.551637057858 partial AUC: 0.0238748040368 Specificity At Sensitivity 0.8: 0.278982951205. my scores are only two digits, are you sure you use the same scoring **harness** for everyone?
@thomas.yu I have exactly the same issue now: "SubjectIds must be part of the gold standard", submission ID: 7878053. Could you please help me with this?
Dear Yohanes, Yes you can assume that everything is ok if you receive the scored submission email. Thank you for your suggestion. I will bring this up internally. Many thanks, Thomas
@thomas.yu thanks, Thomas! Can I assume that everything is ok if I receieved such email? ``` Your submission "submission_scoring.txt" (ID: 7799685) to the Digital Mammography Prediction - Sub-Challenge 1 has been scored: Your submission was missing predictions for 0.000000% of the samples, the confidence values of these samples will be filled in with the median. Your scores are: auc: 0.551637057858 partial AUC: 0.0238748040368 Specificity At Sensitivity 0.8: 0.278982951205. ``` Also IMO it would be very nice if the content of submission file can be put inside the scoring notification email (or any reference to it) as well so that we know exactly which submission file does it related too. Eg: ``` model_state=syn7798989 inference=docker.synapse.org/syn7502921/mik2015-keras-score@sha256:bc1b58a7d07a370ee83b40bde86165e8b4e4380395b3e6e989f86fdd0c70f96c ```
Dear Yohanes, The issue should be fixed. There was a bug in my code. Your two submissions should have been scored. Please kindly let me know if you have any further questions. Best, Thomas

Validation error in submission to Digital Mammography Prediction - Sub-Challenge 1: SubjectIds must be part of the gold standard page is loading…