Hi This is to let you know that there will be an additional delay in opening the leaderboard phase. We're sorry, I know teams are eager to start testing their methods. We're fixing some last issues with our scripts for the automatic scoring of submissions from Synapse. This may take a few days. Best, Daniel

Created by Daniel Marbach daniel.marbach
> does one have to pay sage to run a competition if they want some system-admin support? No, it's usually a collaboration between members of DREAM, Sage, data producers, etc. and the team at Sage is very supportive even if there is no funding. But some challenges have dedicated funding through a grant or sponsors, in which case part may go to Sage and then they could allocate someone specifically to this project and be much more closely involved / spend more time as part of the grant. --daniel
yes, that's what i meant (english problem) thanks
@gyuanfan Thanks, yes we first do a number of checks to see if a given submission is valid. I wasn't sure if I understood your second point though: > 2. whether genes with ids in other networks pop in (a very easy way to cheat the system, right?) You mean whether a module prediction for network A contains genes that are not part of network A? Yes, we do check that. I'm not sure how including gene IDs that are not part of the network would be a way to cheat the system though. As each of the networks is anonymized individually, I don't see how genes from another network could be added. --daniel
thanks daniel. that's very helpful and interesting information. does one have to pay sage to run a competition if they want some system-admin support? i think it would be good to test the scenarios i mentioned above to see if an error is raised (otherwise i would test, thanks).
Dear all Thanks again for your patience. We had some difficulties integrating our scripts with Synapse, but it seems to work now and we are running further tests over the next days (as we don't have dedicated funding for this challenge we don't have a programmer from Sage Bionetworks actively working on this challenge, but the team from Sage is very helpful with support). You might see some testing activity on leaderboard and submission pages -- I added a note that this is just for testing (it seems there is no easy way to test this privately as we cannot make a single page of the wiki private). Best, Daniel
Thanks Yuanfang for clarification...
it's not opened, when it is opened by clicking on any file it would pop up an evaluation queue for this challenge. but i think we really be patient and give the organizers enough time to test leaderboard, then being sorry after a month. e.g. 1. sanity check whether there are overlapping genes among modules. 2. whether genes with ids in other networks pop in (a very easy way to cheat the system, right?) 3. the size of the module 4. multiple hypothesis testing. 5. being able to read in tab and space. 6. file format check... etc... just a few things of many more that they have to implement.
Has the leaderboard started? We haven't recieved any notification. We can see leaderboard for sub-challenge 1 and there is only one team (sarvteam) getting evaluated. Also,there is no link to submit to Module sub-challenge 1. We presume that it is the organizers using it for testing purpose? . Please clarify.
Hi, As there are delays with the opening of the leaderboard, is there any chance for the leaderboard 1 to be open longer than planned, maybe overlapped with the leaderboard 2 phase? Thanks

Opening of leaderboard phase delayed page is loading…