Stijn raised some good points:
_In each round a number of submissions are made. I can, as the last submission, take the best performing clustering from each of my previous submissions (across the different networks) to optimise my NS score. Should I do that, or does the challenge already take this into account? Just to be sure, I am allowed to use different parameters/methods for different networks, within a single submission?_
1. **Resubmitting the best clustering for each network from previous submissions is NOT a good idea because you won't gain any new information** (there's no point in re-scoring the module predictions for a given network, the scoring is deterministic and the result will be identical). It would be a waste of your submission and our computation time. It would of course give you the satisfaction of seeing your name at the top of the leaderboard, but **your ranking in the leaderboard is COMPLETELY IRRELEVANT** for the challenge and will be IGNORED. The sole purpose of the leaderboard is for teams to develop/tune their methods.
2. **There will be a final submission round where every team makes their final submission** (due Oct. 1st, details will be posted soon), which is the only thing that we will look at in the end.
3. **You can use different parameters/methods for the different networks**. I mainly thought of different parameter settings for the same method, I didn't think of the possibility that teams might use different methods. But that's of course a possibility and it is allowed. However, it does raise a bunch of potential issues that I will address in a different post.
Cheers, Daniel
Created by Daniel Marbach daniel.marbach PS: It's certainly a good idea to already now in the leaderboard rounds use different settings for each network in a given submission, as it gives more flexibility to explore which settings work best for a given network.
Drop files to upload
Do NOT resubmit the best clustering from previous submissions to optimize your NS score! page is loading…