Dear organizer,
I am not entirely sure about what has been scored in SC1. Is it only evaluating the top 10 predicted targets for each compound? If none of the top 10 predicted targets are actually true then what kind of score is expected? What if a compound has more than 10 true targets?
I think more information will help us evaluate our submissions in the leaderboard. Thanks.
Best,
Jing
Created by Jing Tang jtjing Hi there,
We are not evaluating based on the top 10 true targets for each compound, rather, we slice out the top 10 predicted targets provided by the participant and evaluate the overlap between that list and the ground truth dataset that we have.
If none of the top 10 predicted targets exist in the truth list, we expect a high p-value (which is converted to a low -log p-value for the leaderboard) compared to a null model.
I hope this clarifies the scoring for you!
Cheers,
Robert