Hi,
I appreciate the organizers to provide the overview in the webinar today.
I have one question about the pKd activity threshold that was discussed in the webinar.
As mentioned in the webinar and as common practice, I believe the pKd~5 should be the threshold for active/inactive pairs. However, in the section 2.4 Scoring of Submissions, it is stated that pKd=7 will be used to binarize the pKd values.
It is a hundred times stricter threshold for active pairs, equivalent to Kd=100 nM (as opposed to Kd=10 uM for pKd=5).
Which one will be used for the actual scoring for the challenge?
Thank you for your time.
Created by Hansaim Lim hansaimlim Thank you for clarifications, @aittokal and @allawayr. Hi @hansaimlim, we are revealing the Round 2 metric after the close of Round 1 to avoid optimization of Round 1 algorithms for any one particular metric. The Round 2 metric has already been selected by the Challenge steering committee and co-organizers. The pKd threshold of 7 is indeed based on the available Kd bioactivity data available in DTC and other resources, and will be used for the F1 score. However, the average AUC score is based one various threshold values over the pKd interval. Thank you @allawayr for clarification. It definitely helps.
By the way, what will be the metric that determines winner of round 2? It' stated that one of the 6 metrics, which is unclear to me. Is there any reason for not specifying the metric?
Thank you again!
Hi @hansaimlim,
To clarify the discussion on today's webinar, a reasonable expectation for *minimum* pK~d~ could be in the range of 4-5. This is based on typical pK~d~ data available in DTC, ChEMBL, or other resources.
However, defining *active* vs inactive is a different threshold. A commonly-used threshold for "active" when discussing pK~d~ is 7 (100 nM). One reason for this threshold is these assays are performed in a cell-free context, and it can commonly take much less drug (orders of magnitude) to have an inhibitory effect in a cell free assay as compared to an assay performed in a biological system. So, if the goal is to (eventually) find a potent molecule in a biological context, a fairly stringent pK~d~ threshold for "active" is required (of course, there many other reasons that a molecule may be active in a cell-free context but inactive in a biological system, but that is beyond the scope of this Challenge).
Also, there are other metrics that do not binarize the predictions, so this is only a factor for a couple of the metrics.
I have tagged @aittokal @anna.cichonska @Guru for their feedback as well.
I hope this helps answer your question!