Hello,
our institutions has strict rules about the release of source code, therefore I would
like to have some clarifications in order to decide whether we are able to participate
to the challenge.
Could you please clarify the meaning of submission requirement #4, i.e. the one saying
"Reproducible Code - For each set of predictions, your code must be provided."?
Specifically:
- is the release of source code mandatory? What about submitting instead an executable,
or pseudocode describing the algorithm (as we did in a previous challenge)?
- who is going to have access to the submitted code?
- what rights are we going to retain on the submitted code?
Thanks,
Mario.
Created by Mario Lauria brickwall thanks solly. i think that makes great sense, to give the full rank.
to announce a so-called winner whose score isn't even top is like telling a sloth you are the fastest sloth in the world; everyone knows what happened and is completely useless. however, a top scorer without a title of winner can still be useful. i think it will be an awesome option for many participants. For this challenge, participants are free to participate in as many or few rounds of the challenge as they see fit. The weighting scheme is defined on the "Submission Requirements and Challenge Scoring" page, including the penalty for missing a round. All participants who submit predictions to at least one phase of the challenge will be scored and all scores will be released on the public scoreboards. The scoreboards provide a public record of challenge submissions, and submissions can not be removed. We will score all submissions but note that failing to complete challenge requirements such as sharing of code, predictor lists and LOOCVs for phases in which they submit predictions will render the participant ineligible to win the challenge or receive challenge incentives such as authorship.
As for withdrawing from papers from previous challenges, I cannot speak to policies for those challenges. You'll need to address the question to the challenge leaders for those particular challenges. may i ask if a participant determines the incentives are unacceptable and/or other unfortunate events happening later, and thus choose not to publish code, will her score be maintained on the leaderboard, or will the leaderboard be manipulated and faked?
thanks a bunch for your feedback
yuanfang
i am asking this question in general, since my previous student who worked with me has left for industry and all the rest projects are all mine too far away my research topic. i am considering withdrawing a couple of the existing paper submissions that i am not happy with.
thanks for your input.
Mario-
Source code is required to be eligible for challenge incentives, and is a requirement in the standard DREAM Challenge rules. In addition to sharing with the challenge organizers, the source code must be shared publicly upon challenge close. If your institution prohibits you from sharing in either manner, we're still happy to score your submissions, but you won't be eligible to win.
Solly