Hi Initially I thought the continuous prediction score would essentially amount to a probability of being classified as high risk (progression in less than 18 months). But after listening to the webminar and the fact that you are thinking of using the concordance index, integrated time depend auc etc now it seems that this actually refers to a risk score from a survival model. Could you clarify how we should interpret this continuous prediction score? Should we be developing 2 models one for the survival (time to event ) outcome and one for the binary high risk flag? or should we only create models for the high risk flag and provide a continuous measure of how likely the person is in the high risk group. If it is the first case then I'm surprise that for the binary outcomes you only require the discrete prediction. Essentially you wont be able to estimate AUCs, etc. It seems to me that you should be asking for 3 predictions from the model: the risk score from survival model, the probability of high risk and the high risk flag. if it is the second case, then you should not be using time dependent AUC or concordance index and instead use AUC or prAUC or some other appropriate measure. This is what is in the wiki right now: "continuous prediction score should be output in column predictionscore. The prediction scores are not assumed to be on any particular scale. Rather, model confidence in assigning high risk should increase monotonically with prediction score--i.e., for two patients, the one with the numerically higher prediction score is at higher risk. " thanks for the clarification DA

Created by exquirentibus veritatem exquirentibus
Dear dreamAnon You should only develop one model per challenge question. The continuous risk score is not necessarily the risk score from a Cox proportion hazard model or other survival model. It can be the probability of progressing before 18 months as you originally intuited. In some other challenges we ask for a continuous risk score and a threshold for dichotomizing it. Here we ask for the continuous risk score and the dichotomized value after your own thresholding. This enable AUC and similar calculation from the continuous score and balanced accuracy, F1 score and others based on the contingency table based off your dichotomized values.

clarification on continuous prediction score page is loading…