now that i have to normalize every output between 0 and 1, i actually don't know how could i possibly do that, as my last step right now i am using linear regression instead of logistic. i am worried some individual may go slightly out the range of all training examples, and then my code would fail.   Then to be safe to be within [0,1] the only way I can think of is to first add a small number and divide it by a bigger number, then I found a prediction of 1.06 might have the same value as 1.07, if only two digits are taken; or similarly 1.006 might have the same value of 1.007 if three digits are taken in the evaluation code. so could you please tell me **how many digits are taken in your evaluation code**?

Created by Yuanfang Guan ???? yuanfang.guan
@yuanfang.guan === What I meant is to use a **global** max value to scale your scores. It can be preset. You don't have to change it from time to time. Would that answer your concern?
@ thefaculty **this is not allowed as specified in other posts**, because one individual's prediction will be affected by another individual. but I plan to tune a new network with softmax instead at the end, i think, that should solve this problem.
@yuanfang.guan @davecg === I have a suggestion: you can simply divide your predicted scores by the max possible score produced by your classifier. That will make all of your entries within the [0, 1] range. Since AUC only cares about ranks, as David pointed out, this will not affect your model's performance at all.
Dear Yuanfang, All the digits are taken in the evaluation code. The rounding only happens for the display on the leaderboard. Does that answer your question? Best, Thomas
Interesting, good to know. Guess we're stuck with adding a logistic function at the end, just a question of how to scale the data first.
@davecg Hi, David Gutman, **they have said explicitly that this is something will make an entry invalid**, because then the prediction value will determined by the other individuals, and the predictions will be different when you predict one at a time and 10 at a time. Actually this is the problem i spent most time on so far. Because in all other competitions using AUC as measurement, obviously we assemble ranks in the end. But since they have said this is not allowed in the entry, otherwise, performance will be better, and it will be much easier.
Since we're being evaluated on auROC and partial auROC only, you could just convert your test scores into a ranking (i.e. 1-100) and distribute them evenly over [0,1] (I.e. Divide by # test samples). Only thing that really matters is the order. Wouldn't be ideal for the final code, but would avoid any numeric issues for now.

evaluation code to how many digits page is loading…