As far as I understand, you do the ranking based on combination of AUPR and AUROC but I want to know how you combine those scores to get the final ranking. What is the formula used to compute the overall ranking score? I would like to see this score that is used to rank the methods in SC1 and SC2 leatherboards as well. Thanks.

Created by Zafer Aydin zaferaydin
Ok thanks for the clarification. To me it still makes more sense to take the average of the score values instead of averaging the ranks, which is prone to rounding effects. For instance there could be only a minor difference between the two AUPRC values but when you rank one of them as 1 and the other as 2 you are increasing the distance. On the other hand there could be a large difference between the two AUROC values but when you rank one of them as 2 and the other as 3 you also modify the distance and make it larger again. This type of ranking puts a greater penalty to minor differences between otherwise indistinguishable methods.
Zafer- By quick glance you are ranked 3rd for AUPR and 2nd for AUROC, so your "score" is 2.5. The team ahead of you is ranked 1st for AUPR and 3rd for AUROC, so their "score" is 2. The rankings are correct. Please note that we average the _ranks_ not the AUCs. Also, please keep in mind that this stage is a collaborative analysis rather than a competition, so there is nothing to be gained by being first. Solly
Ok then my overall score in SC2 is 0.7463, which is better than the score of the jdn team (which is 0.74295) but I am listed as 2nd. Could there still be a bug in the evaluation code? Please show the overall scores for SC1 and SC2. Thanks.
Zafer- The AUPR and AUROC are combined by taking the average of the two ranks. All submissions are ranked by AUPR and also by AUROC. The final rank is based on the average of the two ranks. Solly

Overall ranking score on Leatherboards page is loading…