The log :
STDERR: 2020-04-09T05:39:20.451008348Z c9f212c2013a 2020-04-09 05:39:20,450 MainThread INFO cwltool: Resolved '/var/lib/docker/volumes/workflow_orchestrator_shared/_data/6a31e476-cedc-4288-9bc9-ceaca86a35c3/CTD2-BeatAML-Challenge-master/docker_workflow_sc1.cwl' to 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/6a31e476-cedc-4288-9bc9-ceaca86a35c3/CTD2-BeatAML-Challenge-master/docker_workflow_sc1.cwl'
-packages/toil/leader.py", line 246, in run
STDERR: 2020-04-09T05:49:46.339835238Z raise FailedJobsException(self.config.jobStore, self.toilState.totalFailedJobs, self.jobStore)
STDERR: 2020-04-09T05:49:46.339837441Z toil.leader.FailedJobsException
But these information can't help me to find the problem of my code.
Could you check where the problem is ? **Submission ID: 9702922 (SC1)**
By the way, I didn't find the result of **submission ID 9702923 ( SC2 )** . Can you help me ?
Created by lixin yan yanlixin Hi @fuhrmanj ,
The error you are experiencing is a separate issue from the participant above. With respect to yours, I believe an execution timeout occurred due to a sudden influx of submissions, which consequently resulted in no predictions file being generated (hence your error). We do try to forecast and allocate the computing resources appropriately, but unfortunately, cases like these still pop up from time to time. When it does, we will re-run the submission, which would explain your previous experience. With that said, I will run your latest submission and hope for the best!
Sorry for the inconvenience,
Verena Had the same error message, although docker image worked perfectly on my personal computer. I tested it with mounting the input into the Container via: 'docker run -v :/input '. It worked perfectly and I could get the predictions.csv with 'docker cp :/output/predictions.csv '. There was nothing unusual when checking the file. In addition: I've had the bug before after a submission. Nevertheless after 6 hours I got a new message, that my predictions were scored afterwards, although I did no new submission.
Joshua @yanlixin ,
You may have selected the wrong queue for your second submission, as submission ID 9703507 is attached to the SC1 queue, not SC2. The submission before that (submission ID 9703505) is successfully scored for SC2, as was confirmed by the email you received.
Best,
Verena But I only submit SC2 yesterday. I want to confirm did I got scored both SC1 and SC2? @yanlixin,
--Which queue did you submit to?--
EDIT: it looks like this submission ID is for SC1, not SC2.
Best,
Verena @v.chung
Hi,
Today I submmit SC2 twice. First time I received email told me the submission had been scored.
But I submit SC2 using the same docker file for the second time, I was told that the workflow failded. submission ID 9703507
Could you please check the submission for SC2 is successful or not ?
Synapse:syn21989073
Thank you for your reply ?
I will try to make submission on Leaderboard queues !
By the way, I want to know wheter the validation pahse data will be released to us or not after the challenge is ended ? Hi @yanlixin,
I wanted to follow-up and let you know that the Leaderboard queues have been re-opened! Please note that only one successful submission is accepted per day, so if needed, please continue to use the scoring code shared above to continue scoring your model locally.
Best,
Verena @yanlixin,
No worries! Always happy to help. @v.chung
Ok, thanks a lot !
I understand, sorry for my misunderstanding !
I should study more deeply.
Thank you again ? @yanlixin,
The columns, `vitalStatus` and `overallSurvival`, are for the goldstandard file, not the predictions file. In the [SC2 AUC code](https://www.synapse.org/#!Synapse:syn20940518/discussion/threadId=6886&replyId=21924), `predictions` is a dataframe of the predictions file _and_ the goldstandard file, joined together by `lab_id`.
I hope this helps clear up the misunderstanding?
Verena Hi,
Thank you for your reply, it helps a lot to me !
I'm sorry I have some more questions. **I'm a novice in this field.**
I tried the example of SC2 in github, the predictions.csv only has two columns - 'lab_id' and 'survival' .
However, in the code you shared to me, shows that the predictions.csv should have three columns - 'lab_id' 'vitalStatus' ?overallSurvival?.
I submit the model of SC2, the status is Valid . So I'm a little confusing about the result.
**Thank you very much for your reply.** @yanlixin,
Apologies about that! The logs are only for the Docker runs. However, for your most recent SC1 submission, the "invalid" status is stemming from the generated prediction file instead, which you should have received in an email. If you have not received this email, try checking your spam folder. Additionally, I have also added a "Prediction File Errors" column to the Submission Dashboard so that you may check for prediction errors there.
Best,
Verena Thank you !
I just submit a docker file of SC1, the Prediction File Status is INVALID, the log information can't help me to find the problem, do you know where the preblem is ? @yanlixin,
Unfortunately, the Leaderboard queues are no longer available. You may still locally score your model though! The scoring code for SC1 can be found [here](https://www.synapse.org/#!Synapse:syn20940518/discussion/threadId=6843&replyId=21621), and SC2 [here](https://www.synapse.org/#!Synapse:syn20940518/discussion/threadId=6843&replyId=21645) and [here](https://www.synapse.org/#!Synapse:syn20940518/discussion/threadId=6886&replyId=21924).
Best,
Verena Thank you for your reply, I understand.
So, now can I submit the leaderboard queues ?
I didn't find the leaderboard queues when I submit my docker files.
I want to know the leaderboard score of my models. @yanlixin,
The Express queues do not return scores, as noted in the announcement [here](https://www.synapse.org/#!Synapse:syn20940518/discussion/threadId=6907). The purpose of these queues are so you may check if your model will run successfully against the validation data (since the Validation queues do not return logs).
Note that submitting to the Validation queues also do not return scores.
Hope this helps!
Verena how can I get the score in the leaderboard ? the statut shows that the Prediction File Status is VALIDATED, but I don't know where the score is ?
Dear @yanlixin,
If you are the submitter for your team, you will be able to monitor all submitted submissions (including their current run status and log folder locations) from the [Submission Dashboard](https://www.synapse.org/#!Synapse:syn20940518/wiki/600426). The errors received in the **Workflow Failed** emails aren't always helpful, so we recommend looking through the log files instead!
Hope this helps!
Verena