I can successfully run my scripts via docker on my machine, but they are erroring out when I submit them. I do not understand the log files. Can someone please point out what the actual error is? I've copied my team's log starting where the log file first differs from what I get when I submit the example. ``` STDERR: 2021-02-26T18:24:48.360036797Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/worker_log.txt STDERR: 2021-02-26T18:31:13.783967860Z 1b78d83578db 2021-02-26 18:31:13,783 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj STDERR: 2021-02-26T18:31:13.785162691Z 1b78d83578db 2021-02-26 18:31:13,784 MainThread WARNING toil.leader: The job seems to have left a log file, indicating failure: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj STDERR: 2021-02-26T18:31:13.785550182Z 1b78d83578db 2021-02-26 18:31:13,785 MainThread WARNING toil.leader: Log from job kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj follows: STDERR: 2021-02-26T18:31:13.785560112Z =========> STDERR: 2021-02-26T18:31:13.785564782Z INFO:toil.worker:---TOIL WORKER OUTPUT LOG--- STDERR: 2021-02-26T18:31:13.785568892Z INFO:toil:Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-02-26T18:31:13.785573172Z [job run_synthetic_training_docker.cwl] /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o$ python \ STDERR: 2021-02-26T18:31:13.785577462Z /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o/run_synthetic_training_docker.py \ STDERR: 2021-02-26T18:31:13.785581622Z -s \ STDERR: 2021-02-26T18:31:13.785585042Z 9711075 \ STDERR: 2021-02-26T18:31:13.785588462Z -p \ STDERR: 2021-02-26T18:31:13.785592072Z docker.synapse.org/syn24989427/ryan-squared-q1-take4 \ STDERR: 2021-02-26T18:31:13.785609482Z -d \ STDERR: 2021-02-26T18:31:13.785612842Z sha256:95ff8254f05364d312aff0f915601bc585d4a8311b8d46229c54dceb24bde5f4 \ STDERR: 2021-02-26T18:31:13.785616012Z --training \ STDERR: 2021-02-26T18:31:13.785619072Z --parentid \ STDERR: 2021-02-26T18:31:13.785622082Z syn24995086 \ STDERR: 2021-02-26T18:31:13.785625042Z -c \ STDERR: 2021-02-26T18:31:13.785628052Z /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/tmp04i45yac/stgb5f8eb06-5894-40e0-b11c-c6adfa249af6/.synapseConfig \ STDERR: 2021-02-26T18:31:13.785631332Z -i \ STDERR: 2021-02-26T18:31:13.785634252Z synthetic_omop_covid_q1_train_08-19-2020 \ STDERR: 2021-02-26T18:31:13.785637242Z -q \ STDERR: 2021-02-26T18:31:13.785640152Z 7200 STDERR: 2021-02-26T18:31:13.785643232Z INFO:cwltool:[job run_synthetic_training_docker.cwl] /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o$ python \ STDERR: 2021-02-26T18:31:13.785647712Z /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o/run_synthetic_training_docker.py \ STDERR: 2021-02-26T18:31:13.785651202Z -s \ STDERR: 2021-02-26T18:31:13.785654132Z 9711075 \ STDERR: 2021-02-26T18:31:13.785657072Z -p \ STDERR: 2021-02-26T18:31:13.785659972Z docker.synapse.org/syn24989427/ryan-squared-q1-take4 \ STDERR: 2021-02-26T18:31:13.785663012Z -d \ STDERR: 2021-02-26T18:31:13.785666062Z sha256:95ff8254f05364d312aff0f915601bc585d4a8311b8d46229c54dceb24bde5f4 \ STDERR: 2021-02-26T18:31:13.785669112Z --training \ STDERR: 2021-02-26T18:31:13.785672042Z --parentid \ STDERR: 2021-02-26T18:31:13.785674982Z syn24995086 \ STDERR: 2021-02-26T18:31:13.785678022Z -c \ STDERR: 2021-02-26T18:31:13.785681072Z /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/tmp04i45yac/stgb5f8eb06-5894-40e0-b11c-c6adfa249af6/.synapseConfig \ STDERR: 2021-02-26T18:31:13.785684362Z -i \ STDERR: 2021-02-26T18:31:13.785687362Z synthetic_omop_covid_q1_train_08-19-2020 \ STDERR: 2021-02-26T18:31:13.785690332Z -q \ STDERR: 2021-02-26T18:31:13.785693192Z 7200 STDERR: 2021-02-26T18:31:13.785696182Z Welcome, ehrdreamservice! STDERR: 2021-02-26T18:31:13.785699132Z STDERR: 2021-02-26T18:31:13.785702042Z root STDERR: 2021-02-26T18:31:13.785704932Z training STDERR: 2021-02-26T18:31:13.785707822Z mounting volumes STDERR: 2021-02-26T18:31:13.785710792Z checking for containers STDERR: 2021-02-26T18:31:13.785717522Z running container STDERR: 2021-02-26T18:31:13.785720502Z creating logfile STDERR: 2021-02-26T18:31:13.785723412Z STDERR: 2021-02-26T18:31:13.785726302Z ################################################## STDERR: 2021-02-26T18:31:13.785729302Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:13.785732272Z ################################################## STDERR: 2021-02-26T18:31:13.785735262Z STDERR: 2021-02-26T18:31:13.785738102Z STDERR: 2021-02-26T18:31:13.785740972Z ################################################## STDERR: 2021-02-26T18:31:13.785743952Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:13.785746882Z ################################################## STDERR: 2021-02-26T18:31:13.785749822Z STDERR: 2021-02-26T18:31:13.785752642Z STDERR: 2021-02-26T18:31:13.785755502Z ################################################## STDERR: 2021-02-26T18:31:13.785758442Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:13.785761382Z ################################################## STDERR: 2021-02-26T18:31:13.785764352Z STDERR: 2021-02-26T18:31:13.785767212Z STDERR: 2021-02-26T18:31:13.785770062Z ################################################## STDERR: 2021-02-26T18:31:13.785773102Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:13.785776012Z ################################################## STDERR: 2021-02-26T18:31:13.785778962Z STDERR: 2021-02-26T18:31:13.785781812Z STDERR: 2021-02-26T18:31:13.785784622Z ################################################## STDERR: 2021-02-26T18:31:13.785787612Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:13.785790552Z ################################################## STDERR: 2021-02-26T18:31:13.785793502Z STDERR: 2021-02-26T18:31:13.785796342Z STDERR: 2021-02-26T18:31:13.785799282Z ################################################## STDERR: 2021-02-26T18:31:13.785802272Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:13.785805192Z ################################################## STDERR: 2021-02-26T18:31:13.785808132Z STDERR: 2021-02-26T18:31:13.785811022Z STDERR: 2021-02-26T18:31:13.785813862Z UPGRADE AVAILABLE STDERR: 2021-02-26T18:31:13.785816752Z STDERR: 2021-02-26T18:31:13.785819672Z A more recent version of the Synapse Client (2.2.2) is available. Your version (2.0.0) can be upgraded by typing: STDERR: 2021-02-26T18:31:13.785822942Z pip install --upgrade synapseclient STDERR: 2021-02-26T18:31:13.785825912Z STDERR: 2021-02-26T18:31:13.785828792Z Python Synapse Client version 2.2.2 release notes STDERR: 2021-02-26T18:31:13.785835212Z STDERR: 2021-02-26T18:31:13.785838252Z https://python-docs.synapse.org/build/html/news.html STDERR: 2021-02-26T18:31:13.785841262Z STDERR: 2021-02-26T18:31:13.785844112Z finished training STDERR: 2021-02-26T18:31:13.785847022Z Traceback (most recent call last): STDERR: 2021-02-26T18:31:13.785850532Z File "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o/run_synthetic_training_docker.py", line 222, in
STDERR: 2021-02-26T18:31:13.785854692Z main(syn, args) STDERR: 2021-02-26T18:31:13.785857792Z File "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o/run_synthetic_training_docker.py", line 189, in main STDERR: 2021-02-26T18:31:13.785861742Z raise Exception("No model generated, please check training docker") STDERR: 2021-02-26T18:31:13.785865122Z Exception: No model generated, please check training docker STDERR: 2021-02-26T18:31:13.785868322Z [job run_synthetic_training_docker.cwl] Max memory used: 60MiB STDERR: 2021-02-26T18:31:13.785871422Z INFO:cwltool:[job run_synthetic_training_docker.cwl] Max memory used: 60MiB STDERR: 2021-02-26T18:31:13.785874492Z [job run_synthetic_training_docker.cwl] Job error: STDERR: 2021-02-26T18:31:13.785877523Z ("Error collecting output for parameter 'model':\ncovid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl:70:7: Did not find output file with glob pattern: '['model_files.tar.gz']'", {}) STDERR: 2021-02-26T18:31:13.785881283Z ERROR:cwltool:[job run_synthetic_training_docker.cwl] Job error: STDERR: 2021-02-26T18:31:13.785884343Z ("Error collecting output for parameter 'model':\ncovid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl:70:7: Did not find output file with glob pattern: '['model_files.tar.gz']'", {}) STDERR: 2021-02-26T18:31:13.785887973Z [job run_synthetic_training_docker.cwl] completed permanentFail STDERR: 2021-02-26T18:31:13.785891053Z WARNING:cwltool:[job run_synthetic_training_docker.cwl] completed permanentFail STDERR: 2021-02-26T18:31:13.785894153Z Traceback (most recent call last): STDERR: 2021-02-26T18:31:13.785897163Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-02-26T18:31:13.785900403Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-02-26T18:31:13.785903523Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-02-26T18:31:13.785906733Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-02-26T18:31:13.785909773Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-02-26T18:31:13.785912973Z return self.run(fileStore) STDERR: 2021-02-26T18:31:13.785920183Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-02-26T18:31:13.785923553Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-02-26T18:31:13.785927043Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-02-26T18:31:13.785930013Z ERROR:toil.worker:Exiting the worker because of a failed job on host 1b78d83578db STDERR: 2021-02-26T18:31:13.785933453Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj with ID kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj to 0 STDERR: 2021-02-26T18:31:13.785938133Z <========= STDERR: 2021-02-26T18:31:13.787363305Z 1b78d83578db 2021-02-26 18:31:13,787 MainThread WARNING toil.leader: Job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj with ID kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj is completely failed STDERR: 2021-02-26T18:31:27.733742324Z 1b78d83578db 2021-02-26 18:31:27,733 MainThread INFO toil.leader: Finished toil run with 11 failed jobs. STDERR: 2021-02-26T18:31:27.733843534Z 1b78d83578db 2021-02-26 18:31:27,733 MainThread INFO toil.leader: Failed jobs at end of the run: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.0/cwl/get_docker_config.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.0_cwl_get_docker_config.cwl/instance-rc49swj0 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/validate_docker.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_validate_docker.cwl/instance-47acxh0v 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_submission.cwl/instance-1pufxuqi 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/get_evaluation_id.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_get_evaluation_id.cwl/instance-6bay0oas 'CWLWorkflow' kind-CWLWorkflow/instance-0hxu8qjw 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/dockstore-tool-synapse/v0.2/cwl/synapse-get-tool.cwl' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_dockstore-tool-synapse_v0.2_cwl_synapse-get-tool.cwl/instance-wjbhbpk4 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-1ny56kqv 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/get_dataset.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_get_dataset.cwl/instance-_un7cbxj 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-slizqqx6 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-j74a2nm9 STDERR: 2021-02-26T18:31:27.749631687Z Traceback (most recent call last): STDERR: 2021-02-26T18:31:27.749648667Z File "/usr/local/bin/toil-cwl-runner", line 8, in
STDERR: 2021-02-26T18:31:27.749653487Z sys.exit(main()) STDERR: 2021-02-26T18:31:27.749657027Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 1691, in main STDERR: 2021-02-26T18:31:27.749661367Z outobj = toil.start(wf1) STDERR: 2021-02-26T18:31:27.749664767Z File "/usr/local/lib/python3.6/site-packages/toil/common.py", line 829, in start STDERR: 2021-02-26T18:31:27.749668567Z return self._runMainLoop(rootJobGraph) STDERR: 2021-02-26T18:31:27.749671867Z File "/usr/local/lib/python3.6/site-packages/toil/common.py", line 1120, in _runMainLoop STDERR: 2021-02-26T18:31:27.749675127Z jobCache=self._jobCache).run() STDERR: 2021-02-26T18:31:27.749678327Z File "/usr/local/lib/python3.6/site-packages/toil/leader.py", line 269, in run STDERR: 2021-02-26T18:31:27.749681677Z raise FailedJobsException(self.config.jobStore, self.toilState.totalFailedJobs, self.jobStore) STDERR: 2021-02-26T18:31:27.749692167Z toil.leader.FailedJobsException: The job store 'file:/var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/tmpy8m4kv63' contains 11 failed jobs: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.0/cwl/get_docker_config.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.0_cwl_get_docker_config.cwl/instance-rc49swj0, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/validate_docker.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_validate_docker.cwl/instance-47acxh0v, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_submission.cwl/instance-1pufxuqi, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/get_evaluation_id.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_get_evaluation_id.cwl/instance-6bay0oas, 'CWLWorkflow' kind-CWLWorkflow/instance-0hxu8qjw, 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/dockstore-tool-synapse/v0.2/cwl/synapse-get-tool.cwl' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_dockstore-tool-synapse_v0.2_cwl_synapse-get-tool.cwl/instance-wjbhbpk4, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-1ny56kqv, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/get_dataset.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_get_dataset.cwl/instance-_un7cbxj, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-slizqqx6, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-j74a2nm9 STDERR: 2021-02-26T18:31:27.749717357Z Log from job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj follows: STDERR: 2021-02-26T18:31:27.749721747Z =========> STDERR: 2021-02-26T18:31:27.749724907Z INFO:toil.worker:---TOIL WORKER OUTPUT LOG--- STDERR: 2021-02-26T18:31:27.749728147Z INFO:toil:Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-02-26T18:31:27.749731417Z [job run_synthetic_training_docker.cwl] /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o$ python \ STDERR: 2021-02-26T18:31:27.749735197Z /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o/run_synthetic_training_docker.py \ STDERR: 2021-02-26T18:31:27.749738787Z -s \ STDERR: 2021-02-26T18:31:27.749741767Z 9711075 \ STDERR: 2021-02-26T18:31:27.749744727Z -p \ STDERR: 2021-02-26T18:31:27.749747677Z docker.synapse.org/syn24989427/ryan-squared-q1-take4 \ STDERR: 2021-02-26T18:31:27.749750857Z -d \ STDERR: 2021-02-26T18:31:27.749753867Z sha256:95ff8254f05364d312aff0f915601bc585d4a8311b8d46229c54dceb24bde5f4 \ STDERR: 2021-02-26T18:31:27.749760917Z --training \ STDERR: 2021-02-26T18:31:27.749764087Z --parentid \ STDERR: 2021-02-26T18:31:27.749767027Z syn24995086 \ STDERR: 2021-02-26T18:31:27.749769947Z -c \ STDERR: 2021-02-26T18:31:27.749772857Z /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/tmp04i45yac/stgb5f8eb06-5894-40e0-b11c-c6adfa249af6/.synapseConfig \ STDERR: 2021-02-26T18:31:27.749776197Z -i \ STDERR: 2021-02-26T18:31:27.749779067Z synthetic_omop_covid_q1_train_08-19-2020 \ STDERR: 2021-02-26T18:31:27.749782087Z -q \ STDERR: 2021-02-26T18:31:27.749784947Z 7200 STDERR: 2021-02-26T18:31:27.749788027Z INFO:cwltool:[job run_synthetic_training_docker.cwl] /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o$ python \ STDERR: 2021-02-26T18:31:27.749791897Z /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o/run_synthetic_training_docker.py \ STDERR: 2021-02-26T18:31:27.749795507Z -s \ STDERR: 2021-02-26T18:31:27.749798467Z 9711075 \ STDERR: 2021-02-26T18:31:27.749801457Z -p \ STDERR: 2021-02-26T18:31:27.749804377Z docker.synapse.org/syn24989427/ryan-squared-q1-take4 \ STDERR: 2021-02-26T18:31:27.749807377Z -d \ STDERR: 2021-02-26T18:31:27.749810957Z sha256:95ff8254f05364d312aff0f915601bc585d4a8311b8d46229c54dceb24bde5f4 \ STDERR: 2021-02-26T18:31:27.749814067Z --training \ STDERR: 2021-02-26T18:31:27.749816997Z --parentid \ STDERR: 2021-02-26T18:31:27.749819937Z syn24995086 \ STDERR: 2021-02-26T18:31:27.749822807Z -c \ STDERR: 2021-02-26T18:31:27.749825707Z /var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/tmp04i45yac/stgb5f8eb06-5894-40e0-b11c-c6adfa249af6/.synapseConfig \ STDERR: 2021-02-26T18:31:27.749829207Z -i \ STDERR: 2021-02-26T18:31:27.749832127Z synthetic_omop_covid_q1_train_08-19-2020 \ STDERR: 2021-02-26T18:31:27.749835137Z -q \ STDERR: 2021-02-26T18:31:27.749838197Z 7200 STDERR: 2021-02-26T18:31:27.749841127Z Welcome, ehrdreamservice! STDERR: 2021-02-26T18:31:27.749844047Z STDERR: 2021-02-26T18:31:27.749846947Z root STDERR: 2021-02-26T18:31:27.749849787Z training STDERR: 2021-02-26T18:31:27.749853027Z mounting volumes STDERR: 2021-02-26T18:31:27.749856247Z checking for containers STDERR: 2021-02-26T18:31:27.749859217Z running container STDERR: 2021-02-26T18:31:27.749865427Z creating logfile STDERR: 2021-02-26T18:31:27.749868467Z STDERR: 2021-02-26T18:31:27.749871367Z ################################################## STDERR: 2021-02-26T18:31:27.749874357Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:27.749877397Z ################################################## STDERR: 2021-02-26T18:31:27.749880367Z STDERR: 2021-02-26T18:31:27.749883167Z STDERR: 2021-02-26T18:31:27.749885987Z ################################################## STDERR: 2021-02-26T18:31:27.749888967Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:27.749891897Z ################################################## STDERR: 2021-02-26T18:31:27.749894907Z STDERR: 2021-02-26T18:31:27.749897737Z STDERR: 2021-02-26T18:31:27.749900607Z ################################################## STDERR: 2021-02-26T18:31:27.749903568Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:27.749906488Z ################################################## STDERR: 2021-02-26T18:31:27.749909418Z STDERR: 2021-02-26T18:31:27.749912248Z STDERR: 2021-02-26T18:31:27.749915068Z ################################################## STDERR: 2021-02-26T18:31:27.749918108Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:27.749921288Z ################################################## STDERR: 2021-02-26T18:31:27.749924228Z STDERR: 2021-02-26T18:31:27.749927018Z STDERR: 2021-02-26T18:31:27.749929818Z ################################################## STDERR: 2021-02-26T18:31:27.749932728Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:27.749935668Z ################################################## STDERR: 2021-02-26T18:31:27.749938718Z STDERR: 2021-02-26T18:31:27.749941538Z STDERR: 2021-02-26T18:31:27.749944328Z ################################################## STDERR: 2021-02-26T18:31:27.749947268Z Uploading file to Synapse storage STDERR: 2021-02-26T18:31:27.749950178Z ################################################## STDERR: 2021-02-26T18:31:27.749953128Z STDERR: 2021-02-26T18:31:27.749955918Z STDERR: 2021-02-26T18:31:27.749958738Z UPGRADE AVAILABLE STDERR: 2021-02-26T18:31:27.749961588Z STDERR: 2021-02-26T18:31:27.749964438Z A more recent version of the Synapse Client (2.2.2) is available. Your version (2.0.0) can be upgraded by typing: STDERR: 2021-02-26T18:31:27.749967548Z pip install --upgrade synapseclient STDERR: 2021-02-26T18:31:27.749970458Z STDERR: 2021-02-26T18:31:27.749973338Z Python Synapse Client version 2.2.2 release notes STDERR: 2021-02-26T18:31:27.749976278Z STDERR: 2021-02-26T18:31:27.749979128Z https://python-docs.synapse.org/build/html/news.html STDERR: 2021-02-26T18:31:27.749985568Z STDERR: 2021-02-26T18:31:27.749988568Z finished training STDERR: 2021-02-26T18:31:27.749991468Z Traceback (most recent call last): STDERR: 2021-02-26T18:31:27.749994468Z File "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o/run_synthetic_training_docker.py", line 222, in
STDERR: 2021-02-26T18:31:27.749999558Z main(syn, args) STDERR: 2021-02-26T18:31:27.750002638Z File "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/node-b185fe19-2ed1-4c40-aa0e-e6729ed6e9cf-79372d8564c8545262975c46157f1482/tmpdwpwio16/e141b774-a0b6-4748-b2ac-f4fecefb3957/t49vm3mns/tmp-out_htfmq5o/run_synthetic_training_docker.py", line 189, in main STDERR: 2021-02-26T18:31:27.750006368Z raise Exception("No model generated, please check training docker") STDERR: 2021-02-26T18:31:27.750009608Z Exception: No model generated, please check training docker STDERR: 2021-02-26T18:31:27.750012608Z [job run_synthetic_training_docker.cwl] Max memory used: 60MiB STDERR: 2021-02-26T18:31:27.750015628Z INFO:cwltool:[job run_synthetic_training_docker.cwl] Max memory used: 60MiB STDERR: 2021-02-26T18:31:27.750018748Z [job run_synthetic_training_docker.cwl] Job error: STDERR: 2021-02-26T18:31:27.750021768Z ("Error collecting output for parameter 'model':\ncovid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl:70:7: Did not find output file with glob pattern: '['model_files.tar.gz']'", {}) STDERR: 2021-02-26T18:31:27.750025588Z ERROR:cwltool:[job run_synthetic_training_docker.cwl] Job error: STDERR: 2021-02-26T18:31:27.750028588Z ("Error collecting output for parameter 'model':\ncovid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl:70:7: Did not find output file with glob pattern: '['model_files.tar.gz']'", {}) STDERR: 2021-02-26T18:31:27.750032168Z [job run_synthetic_training_docker.cwl] completed permanentFail STDERR: 2021-02-26T18:31:27.750035158Z WARNING:cwltool:[job run_synthetic_training_docker.cwl] completed permanentFail STDERR: 2021-02-26T18:31:27.750038218Z Traceback (most recent call last): STDERR: 2021-02-26T18:31:27.750041208Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-02-26T18:31:27.750044478Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-02-26T18:31:27.750047558Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-02-26T18:31:27.750050748Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-02-26T18:31:27.750053698Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-02-26T18:31:27.750056878Z return self.run(fileStore) STDERR: 2021-02-26T18:31:27.750059788Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-02-26T18:31:27.750066548Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-02-26T18:31:27.750069688Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-02-26T18:31:27.750072658Z ERROR:toil.worker:Exiting the worker because of a failed job on host 1b78d83578db STDERR: 2021-02-26T18:31:27.750076588Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/e0694edf-f9d3-45c1-b3db-d560baff6132/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj with ID kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_e0694edf-f9d3-45c1-b3db-d560baff6132_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-46b518oj to 0 STDERR: 2021-02-26T18:31:27.750081208Z <========= ```
Created by
Ryan Melvin rmelvin
Drop files to upload
Your web browser must have JavaScript enabled in order for this application to display correctly.
If you are an automated web crawler from a search engine, follow this
AJAX application crawl link
Question 1 Submission Error
page is loading…