In the "2 - Challenge Questions and Scoring" wiki, there are points awarded for (1) the predicted profile of the compound and (2) the compound itself.
It is unclear how (1) will assessed: are the users supposed to provide a predicted profile and score themselves? I did not see this in the submission file nor in the instructions.
Moreover, how will the admins prevent the scenario that predicted profile will be "adjusted" so as to achieve maximal scoring ("of couuurse my compound has the desired profile!")?
Additionally, since this is an exploration of methods, different teams may have different criteria/scoring functions for what constitutes a "binder" or "non-binder". How will this be handled in the scoring system of (1)?
Or, is (1) scored only after experimental testing, and teams are selected for evaluation based on (2) and a combination of innovation in methods?
Sorry, if this was covered in the webinar. I came a bit late to the game and the instructions confused me a bit!
Any clarification would help greatly.
Created by Stephen Capuzzi unc2015 Thanks @kkdang for the clarification. This was definitely not clear from the start. In that case, I think it would be reasonable to change PAK3 to desired instead of required.
I would be really surprised if anybody finds PAK1/PAK3 selective ligands, looking at the kinase domain we can see why;
${imageLink?synapseId=syn11896956&align=None&scale=100&responsive=true}
There are 9 residues different of which 8 are almost the same (in terms of properties)
To make things worse, none of these residues is anywhere near the orthosteric binding site.
${imageLink?synapseId=syn11896954&align=None&scale=100&responsive=true} Thanks for the questions.
Required elements are all required, so submissions that don't meet the required elements get a score of zero. Indeed this makes challenge question 2 very difficult, which is why this challenge is interested in stimulating novel methods. So how does this work for off targets?
e.g. for challenge 2 this would be really challenging;
PAK1 (required on-target) and PAK3 (required off-target) are almost identical. So this makes challenge 2 more or less impossible? (all compounds will get 0 points?) Dear @kkdang
If a compound has only one of the required elements, will the score be zero?
For instance, if a compound binds to RET (M918T) and binds to MKNK1, is its score zero? Dear @MMochizuki,
Compounds that do not have the required elements will get a score of zero. I have a question on scoring system, too.
For example, please consider assay result of a compound is below:
The compound doesn't bind RET (M918T).
The compound binds all of BRAF, SRC, and S6K.
The compound binds MKNK1.
The compound doesn't binds any of TTK, ErK8, PDK1, and PAK3.
This result satisfies all of "Desired" behaviors, but doesn't any of "Required" ones, according to description on "Challenge Questions and Scoring".
If "Required" behavior(s) were preconditions of points of "Desired" behaviors, I think the score would be 0.
Otherwise, the score would be 9 + 4 = 13.
Which is the actual score in this case?
In addition, how about a case that only one of "Required" behaviors was satisfied? Dear @unc2015,
The point score for the profile of the compound (i.e. its binding activity) will be determined only after testing by the external lab. The point score will be calculated by the challenge scoring system based on these results. We do not expect (or accept) scores submitted by participants.
The additional elements in scoring (patentability, drug-like properties, novelty, ability to pass CNS) will be evaluated by the challenge scoring system to produce a score. Still unclear as to how scoring and ranking works. Any help would be appreciated
Drop files to upload
Scoring and Ranking Clarification page is loading…