A high score in both fast and slow will mean confidence of prediction
Created by Thomas Yu thomas.yu The submission example was provided to standardize the format of submissions so you should not use the actual probabilities in the example. AUROC and AUPRC are estimated using a standard approach so ensure that higher probabilities correspond to "FAST". If you rescale e.g. in the example you provided did you mean your "SLOW" would be anything <0.60 as your "FAST" is 0.8 to 0.6? If so, then that is fine. But the cut-off is your choice and should not be based on the submission example.
I hope this helps. Thanks,
Geoffrey Thanks for your reply. Then could you please explain a little bit about how you calculate the AUROC and AUPRC? Cause I noticed that one sample with 0.92 probability is classified as SLOW and another one with 0.2 is FAST in the submission example you provided for subchallenge 2. I don't really understand how to calculate the probability. I thought that 0.5 was the threshold to determine the SLOW or FAST but it seems like not the case. If I rescale my probability, for example, from FAST:0.8 to FAST:0.6 and SLOW:0.1 to 0.8, are they what you need for the subchallenge 2?
If I didn't make it clear enough, please feel free to post any question.
Thanks,
Di Dear All,
Thank you for your question. We apologize for not being clear.
To be clear, higher probabilities should be associated with 'FAST'. Since 'higher' is relative, we also asked that the submission include the predicted categorical value "SLOW" or "FAST" (as the second column of your submission file).
We hope this addresses your concern.
Malaria DREAM challenge organizers I am confused about this as well. Clarification would be helpful.
Thank you,
Nicola In the "prepare a submission" part, it shows that "the third to the confidence score (between 0-1) for your prediction where higher probabilities should be associated with "fast"". I thought it means high probability always means "FAST" and low one means "SLOW". However, I just saw this question and it seems to be totally opposite with the description in the "prepare a submission" part. May I ask which one I should base on?
Can anyone help me?
Drop files to upload
Webinar 1: For subch2, how does fast/slow prediction relate to the confidence score? page is loading…