Anybody has any thoughts on what the weights could be? ........
Created by dcentmakeover @allawayr i know this has come up before, so feel free to ignore, but any chance the weights could be made public ?Thanks I do not have the slightest idea of the weights, and I am concerned that they have been chosen arbitrarily and that this would skew the contest results.
I think the initial concerns that the organizers have expressed, that some contestants may "play" the scoring metric, have now shown to be unfounded.
There are now several contestants with scores significantly better than the "baseline model", and that are very close to each other.
The fact that the ranking is based on an unrevealed metric, makes it very difficult to optimize further our solutions, other than selecting solutions that score better on the test set, meaning that we all overfit on this test set.
To my opinion, beside of being unfair, encouraging participants to overfit because the scoring metric is kept secret is a very poor way of getting the best solution.
The organizers should level-out the competition and reveal what they consider to be a good solution, by making public their metric formula and weights.