Dear Organizers, I am repeatedly getting this error. Although I have followed all the instructions as suggested. The orientation and the origin of the segmented volumes provided during training perfectly match with our predicted segmented volumes. Please help as we are getting this error repeatedly. STDERR: 2021-08-12T05:32:36.790941950Z 72e686b9bf5d 2021-08-12 05:32:36,790 MainThread INFO cwltool: Resolved '/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/validation-phase-workflow.cwl' to 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/validation-phase-workflow.cwl' STDERR: 2021-08-12T05:32:41.690069492Z 72e686b9bf5d 2021-08-12 05:32:41,689 MainThread WARNING cwltool: Workflow checker warning: STDERR: 2021-08-12T05:32:41.690100065Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:55:9: Source 'filepath' of type STDERR: 2021-08-12T05:32:41.690104915Z ["null", "File"] may be STDERR: 2021-08-12T05:32:41.690109426Z incompatible STDERR: 2021-08-12T05:32:41.690113237Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:135:9: with sink 'input_file' of STDERR: 2021-08-12T05:32:41.690116749Z type "File" STDERR: 2021-08-12T05:32:41.690120255Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:55:9: Source 'filepath' of type STDERR: 2021-08-12T05:32:41.690124294Z ["null", "File"] may be STDERR: 2021-08-12T05:32:41.690128070Z incompatible STDERR: 2021-08-12T05:32:41.690131706Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:75:9: with sink 'input_file' of STDERR: 2021-08-12T05:32:41.690135664Z type "File" STDERR: 2021-08-12T05:32:43.034776622Z 72e686b9bf5d 2021-08-12 05:32:43,034 MainThread INFO toil: Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-12T05:32:43.918151822Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpylmqw17q/worker_log.txt STDERR: 2021-08-12T05:32:44.389107771Z 72e686b9bf5d 2021-08-12 05:32:44,388 MainThread INFO toil.leader: 0 jobs are running, 0 jobs are issued and waiting to run STDERR: 2021-08-12T05:32:44.390181241Z 72e686b9bf5d 2021-08-12 05:32:44,390 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-jq16tsot with job batch system ID: 1 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:32:44.390492279Z 72e686b9bf5d 2021-08-12 05:32:44,390 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/set_permissions.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_set_permissions.cwl/instance-6l6u2t2o with job batch system ID: 2 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:32:44.390624461Z 72e686b9bf5d 2021-08-12 05:32:44,390 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-26ce3171 with job batch system ID: 3 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:32:45.523927668Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmp37g8mxyd/worker_log.txt STDERR: 2021-08-12T05:32:45.594645336Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpfklxbzwl/worker_log.txt STDERR: 2021-08-12T05:32:45.636524627Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpnc0uxlq4/worker_log.txt STDERR: 2021-08-12T05:34:11.506354500Z 72e686b9bf5d 2021-08-12 05:34:11,504 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-26ce3171 STDERR: 2021-08-12T05:34:21.060058777Z 72e686b9bf5d 2021-08-12 05:34:21,059 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/set_permissions.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_set_permissions.cwl/instance-6l6u2t2o STDERR: 2021-08-12T05:34:25.790527959Z 72e686b9bf5d 2021-08-12 05:34:25,790 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-jq16tsot STDERR: 2021-08-12T05:34:25.791208128Z 72e686b9bf5d 2021-08-12 05:34:25,791 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_validate.cwl/instance-lj4xk9sk with job batch system ID: 4 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:34:26.882542398Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpuz0kk16u/worker_log.txt STDERR: 2021-08-12T05:35:02.138855524Z 72e686b9bf5d 2021-08-12 05:35:02,138 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_validate.cwl/instance-lj4xk9sk STDERR: 2021-08-12T05:35:02.139821958Z 72e686b9bf5d 2021-08-12 05:35:02,139 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-017j28jr with job batch system ID: 5 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:35:02.140007143Z 72e686b9bf5d 2021-08-12 05:35:02,139 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-_f3hscla with job batch system ID: 6 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:35:03.201838969Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmp3ur1u1s7/worker_log.txt STDERR: 2021-08-12T05:35:03.288913968Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpphbo6gfw/worker_log.txt STDERR: 2021-08-12T05:37:07.135025469Z 72e686b9bf5d 2021-08-12 05:37:07,134 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-017j28jr STDERR: 2021-08-12T05:37:11.887514211Z 72e686b9bf5d 2021-08-12 05:37:11,887 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-_f3hscla STDERR: 2021-08-12T05:37:11.888024002Z 72e686b9bf5d 2021-08-12 05:37:11,887 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-qf37xcst with job batch system ID: 7 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:37:12.763118791Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmp671b5vl4/worker_log.txt STDERR: 2021-08-12T05:37:13.270663137Z 72e686b9bf5d 2021-08-12 05:37:13,270 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-qf37xcst STDERR: 2021-08-12T05:37:13.271460552Z 72e686b9bf5d 2021-08-12 05:37:13,271 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_score.cwl/instance-9dwzxjp0 with job batch system ID: 8 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:37:14.197733081Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmp07gxzkr2/worker_log.txt STDERR: 2021-08-12T05:38:20.488784154Z 72e686b9bf5d 2021-08-12 05:38:20,488 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_score.cwl/instance-9dwzxjp0 STDERR: 2021-08-12T05:38:20.489825935Z 72e686b9bf5d 2021-08-12 05:38:20,489 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_score_email.cwl/instance-pcoylpfp with job batch system ID: 9 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:38:20.490031424Z 72e686b9bf5d 2021-08-12 05:38:20,489 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-gh9f87_z with job batch system ID: 10 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:38:21.715111758Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpwxxibwfr/worker_log.txt STDERR: 2021-08-12T05:38:21.741329036Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpqngm78ju/worker_log.txt STDERR: 2021-08-12T05:40:09.551837504Z 72e686b9bf5d 2021-08-12 05:40:09,551 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-gh9f87_z STDERR: 2021-08-12T05:40:11.772967795Z 72e686b9bf5d 2021-08-12 05:40:11,772 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_score_email.cwl/instance-pcoylpfp STDERR: 2021-08-12T05:40:11.773521476Z 72e686b9bf5d 2021-08-12 05:40:11,773 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv with job batch system ID: 11 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-12T05:40:12.885026773Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpgz0t8hol/worker_log.txt STDERR: 2021-08-12T05:40:13.355592260Z 72e686b9bf5d 2021-08-12 05:40:13,354 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv STDERR: 2021-08-12T05:40:13.356339571Z 72e686b9bf5d 2021-08-12 05:40:13,356 MainThread WARNING toil.leader: The job seems to have left a log file, indicating failure: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv STDERR: 2021-08-12T05:40:13.356609201Z 72e686b9bf5d 2021-08-12 05:40:13,356 MainThread WARNING toil.leader: Log from job kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv follows: STDERR: 2021-08-12T05:40:13.356619068Z =========> STDERR: 2021-08-12T05:40:13.356622790Z INFO:toil.worker:---TOIL WORKER OUTPUT LOG--- STDERR: 2021-08-12T05:40:13.356626548Z INFO:toil:Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-12T05:40:13.356629910Z Failed to evaluate expression: STDERR: 2021-08-12T05:40:13.356633288Z Expression evaluation error: STDERR: 2021-08-12T05:40:13.356636420Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-12T05:40:13.356639753Z script was: STDERR: 2021-08-12T05:40:13.356659777Z 01 "use strict"; STDERR: 2021-08-12T05:40:13.356663150Z 02 var inputs = { STDERR: 2021-08-12T05:40:13.356666278Z 03 "status": "INVALID", STDERR: 2021-08-12T05:40:13.356670145Z 04 "previous_annotation_finished": true, STDERR: 2021-08-12T05:40:13.356673555Z 05 "previous_email_finished": true STDERR: 2021-08-12T05:40:13.356676928Z 06 }; STDERR: 2021-08-12T05:40:13.356680095Z 07 var self = null; STDERR: 2021-08-12T05:40:13.356683288Z 08 var runtime = { STDERR: 2021-08-12T05:40:13.356686398Z 09 "cores": 1, STDERR: 2021-08-12T05:40:13.356689652Z 10 "ram": 100.0, STDERR: 2021-08-12T05:40:13.356692877Z 11 "tmpdirSize": 1024, STDERR: 2021-08-12T05:40:13.356696340Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-12T05:40:13.356699891Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/tmpbk8zf42v", STDERR: 2021-08-12T05:40:13.356703674Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpgz0t8hol/562f2bc6-52c9-4d9b-9379-50b32531b83a/t8f_k9xm1/tmp-outju6zncpx" STDERR: 2021-08-12T05:40:13.356708120Z 15 }; STDERR: 2021-08-12T05:40:13.356711250Z 16 (function(){ STDERR: 2021-08-12T05:40:13.356714464Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-12T05:40:13.356717763Z 18 throw 'invalid submission'; STDERR: 2021-08-12T05:40:13.356720983Z 19 } else { STDERR: 2021-08-12T05:40:13.356724211Z 20 return {finished: true}; STDERR: 2021-08-12T05:40:13.356727396Z 21 } STDERR: 2021-08-12T05:40:13.356742107Z 22 })() STDERR: 2021-08-12T05:40:13.356745224Z stdout was: '' STDERR: 2021-08-12T05:40:13.356753879Z stderr was: 'invalid submission' STDERR: 2021-08-12T05:40:13.356757073Z STDERR: 2021-08-12T05:40:13.356760448Z WARNING:cwltool:Failed to evaluate expression: STDERR: 2021-08-12T05:40:13.356763625Z Expression evaluation error: STDERR: 2021-08-12T05:40:13.356766665Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-12T05:40:13.356769744Z script was: STDERR: 2021-08-12T05:40:13.356772716Z 01 "use strict"; STDERR: 2021-08-12T05:40:13.356775807Z 02 var inputs = { STDERR: 2021-08-12T05:40:13.356778781Z 03 "status": "INVALID", STDERR: 2021-08-12T05:40:13.356782504Z 04 "previous_annotation_finished": true, STDERR: 2021-08-12T05:40:13.356785909Z 05 "previous_email_finished": true STDERR: 2021-08-12T05:40:13.356789090Z 06 }; STDERR: 2021-08-12T05:40:13.356792172Z 07 var self = null; STDERR: 2021-08-12T05:40:13.356795137Z 08 var runtime = { STDERR: 2021-08-12T05:40:13.356815388Z 09 "cores": 1, STDERR: 2021-08-12T05:40:13.356818530Z 10 "ram": 100.0, STDERR: 2021-08-12T05:40:13.356821714Z 11 "tmpdirSize": 1024, STDERR: 2021-08-12T05:40:13.356825274Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-12T05:40:13.356828620Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/tmpbk8zf42v", STDERR: 2021-08-12T05:40:13.356833058Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpgz0t8hol/562f2bc6-52c9-4d9b-9379-50b32531b83a/t8f_k9xm1/tmp-outju6zncpx" STDERR: 2021-08-12T05:40:13.356837167Z 15 }; STDERR: 2021-08-12T05:40:13.356840196Z 16 (function(){ STDERR: 2021-08-12T05:40:13.356843346Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-12T05:40:13.356846709Z 18 throw 'invalid submission'; STDERR: 2021-08-12T05:40:13.356849838Z 19 } else { STDERR: 2021-08-12T05:40:13.356852941Z 20 return {finished: true}; STDERR: 2021-08-12T05:40:13.356856023Z 21 } STDERR: 2021-08-12T05:40:13.356859099Z 22 })() STDERR: 2021-08-12T05:40:13.356862139Z stdout was: '' STDERR: 2021-08-12T05:40:13.356865457Z stderr was: 'invalid submission' STDERR: 2021-08-12T05:40:13.356868574Z STDERR: 2021-08-12T05:40:13.356871635Z Traceback (most recent call last): STDERR: 2021-08-12T05:40:13.356874800Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-08-12T05:40:13.356878353Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-08-12T05:40:13.356881674Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-08-12T05:40:13.356885155Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-08-12T05:40:13.356891267Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-08-12T05:40:13.356894923Z return self.run(fileStore) STDERR: 2021-08-12T05:40:13.356898119Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-08-12T05:40:13.356901781Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-08-12T05:40:13.356905103Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-08-12T05:40:13.356908441Z ERROR:toil.worker:Exiting the worker because of a failed job on host 72e686b9bf5d STDERR: 2021-08-12T05:40:13.356911944Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv to 0 STDERR: 2021-08-12T05:40:13.356917108Z <========= STDERR: 2021-08-12T05:40:13.357031175Z 72e686b9bf5d 2021-08-12 05:40:13,356 MainThread WARNING toil.leader: Job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv is completely failed STDERR: 2021-08-12T05:40:30.731705085Z 72e686b9bf5d 2021-08-12 05:40:30,731 MainThread INFO toil.leader: Finished toil run with 11 failed jobs. STDERR: 2021-08-12T05:40:30.732047583Z 72e686b9bf5d 2021-08-12 05:40:30,731 MainThread INFO toil.leader: Failed jobs at end of the run: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-qf37xcst 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-017j28jr 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-jq16tsot 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_score_email.cwl/instance-pcoylpfp 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-26ce3171 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-_f3hscla 'CWLWorkflow' kind-CWLWorkflow/instance-aj9j6sn4 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_score.cwl/instance-9dwzxjp0 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-gh9f87_z 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_validate.cwl/instance-lj4xk9sk STDERR: 2021-08-12T05:40:30.746547315Z Traceback (most recent call last): STDERR: 2021-08-12T05:40:30.746576718Z File "/usr/local/bin/toil-cwl-runner", line 8, in STDERR: 2021-08-12T05:40:30.746581839Z sys.exit(main()) STDERR: 2021-08-12T05:40:30.746585634Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 1691, in main STDERR: 2021-08-12T05:40:30.746589408Z outobj = toil.start(wf1) STDERR: 2021-08-12T05:40:30.746592783Z File "/usr/local/lib/python3.6/site-packages/toil/common.py", line 829, in start STDERR: 2021-08-12T05:40:30.746596029Z return self._runMainLoop(rootJobGraph) STDERR: 2021-08-12T05:40:30.746599496Z File "/usr/local/lib/python3.6/site-packages/toil/common.py", line 1120, in _runMainLoop STDERR: 2021-08-12T05:40:30.746603151Z jobCache=self._jobCache).run() STDERR: 2021-08-12T05:40:30.746606467Z File "/usr/local/lib/python3.6/site-packages/toil/leader.py", line 269, in run STDERR: 2021-08-12T05:40:30.746609784Z raise FailedJobsException(self.config.jobStore, self.toilState.totalFailedJobs, self.jobStore) STDERR: 2021-08-12T05:40:30.746616769Z toil.leader.FailedJobsException: The job store 'file:/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/tmpsiuez_vp' contains 11 failed jobs: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-qf37xcst, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-017j28jr, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-jq16tsot, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_score_email.cwl/instance-pcoylpfp, 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-26ce3171, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-_f3hscla, 'CWLWorkflow' kind-CWLWorkflow/instance-aj9j6sn4, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_score.cwl/instance-9dwzxjp0, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-gh9f87_z, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_daa1c417-0f32-49be-a8af-831207d965ed_brats-dream-challenge-infra-main_validate.cwl/instance-lj4xk9sk STDERR: 2021-08-12T05:40:30.746640041Z Log from job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv follows: STDERR: 2021-08-12T05:40:30.746643935Z =========> STDERR: 2021-08-12T05:40:30.746646805Z INFO:toil.worker:---TOIL WORKER OUTPUT LOG--- STDERR: 2021-08-12T05:40:30.746650592Z INFO:toil:Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-12T05:40:30.746653664Z Failed to evaluate expression: STDERR: 2021-08-12T05:40:30.746780328Z Expression evaluation error: STDERR: 2021-08-12T05:40:30.746789266Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-12T05:40:30.746792896Z script was: STDERR: 2021-08-12T05:40:30.746795908Z 01 "use strict"; STDERR: 2021-08-12T05:40:30.746799161Z 02 var inputs = { STDERR: 2021-08-12T05:40:30.746802216Z 03 "status": "INVALID", STDERR: 2021-08-12T05:40:30.746805432Z 04 "previous_annotation_finished": true, STDERR: 2021-08-12T05:40:30.746809026Z 05 "previous_email_finished": true STDERR: 2021-08-12T05:40:30.746811984Z 06 }; STDERR: 2021-08-12T05:40:30.746815329Z 07 var self = null; STDERR: 2021-08-12T05:40:30.746818121Z 08 var runtime = { STDERR: 2021-08-12T05:40:30.746821453Z 09 "cores": 1, STDERR: 2021-08-12T05:40:30.746824289Z 10 "ram": 100.0, STDERR: 2021-08-12T05:40:30.746828260Z 11 "tmpdirSize": 1024, STDERR: 2021-08-12T05:40:30.746831699Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-12T05:40:30.746835182Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/tmpbk8zf42v", STDERR: 2021-08-12T05:40:30.746844726Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpgz0t8hol/562f2bc6-52c9-4d9b-9379-50b32531b83a/t8f_k9xm1/tmp-outju6zncpx" STDERR: 2021-08-12T05:40:30.746849099Z 15 }; STDERR: 2021-08-12T05:40:30.746851896Z 16 (function(){ STDERR: 2021-08-12T05:40:30.746855299Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-12T05:40:30.746858455Z 18 throw 'invalid submission'; STDERR: 2021-08-12T05:40:30.746861866Z 19 } else { STDERR: 2021-08-12T05:40:30.746865062Z 20 return {finished: true}; STDERR: 2021-08-12T05:40:30.746868030Z 21 } STDERR: 2021-08-12T05:40:30.746871569Z 22 })() STDERR: 2021-08-12T05:40:30.746874462Z stdout was: '' STDERR: 2021-08-12T05:40:30.746878220Z stderr was: 'invalid submission' STDERR: 2021-08-12T05:40:30.746881334Z STDERR: 2021-08-12T05:40:30.746884859Z WARNING:cwltool:Failed to evaluate expression: STDERR: 2021-08-12T05:40:30.746888040Z Expression evaluation error: STDERR: 2021-08-12T05:40:30.746891679Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-12T05:40:30.746894556Z script was: STDERR: 2021-08-12T05:40:30.746897877Z 01 "use strict"; STDERR: 2021-08-12T05:40:30.746900862Z 02 var inputs = { STDERR: 2021-08-12T05:40:30.746903744Z 03 "status": "INVALID", STDERR: 2021-08-12T05:40:30.746907456Z 04 "previous_annotation_finished": true, STDERR: 2021-08-12T05:40:30.746910704Z 05 "previous_email_finished": true STDERR: 2021-08-12T05:40:30.746913777Z 06 }; STDERR: 2021-08-12T05:40:30.746916597Z 07 var self = null; STDERR: 2021-08-12T05:40:30.746920384Z 08 var runtime = { STDERR: 2021-08-12T05:40:30.746923219Z 09 "cores": 1, STDERR: 2021-08-12T05:40:30.746927231Z 10 "ram": 100.0, STDERR: 2021-08-12T05:40:30.746930094Z 11 "tmpdirSize": 1024, STDERR: 2021-08-12T05:40:30.746934051Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-12T05:40:30.746937247Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/tmpbk8zf42v", STDERR: 2021-08-12T05:40:30.746941209Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/daa1c417-0f32-49be-a8af-831207d965ed/node-865c3b6d-4303-47ab-8089-25e0d1363a08-16349ee66b573b0d6cff3dbc6912349f/tmpgz0t8hol/562f2bc6-52c9-4d9b-9379-50b32531b83a/t8f_k9xm1/tmp-outju6zncpx" STDERR: 2021-08-12T05:40:30.746944836Z 15 }; STDERR: 2021-08-12T05:40:30.746947901Z 16 (function(){ STDERR: 2021-08-12T05:40:30.746950876Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-12T05:40:30.746953904Z 18 throw 'invalid submission'; STDERR: 2021-08-12T05:40:30.746957213Z 19 } else { STDERR: 2021-08-12T05:40:30.746960478Z 20 return {finished: true}; STDERR: 2021-08-12T05:40:30.746967632Z 21 } STDERR: 2021-08-12T05:40:30.746971502Z 22 })() STDERR: 2021-08-12T05:40:30.746974330Z stdout was: '' STDERR: 2021-08-12T05:40:30.746977860Z stderr was: 'invalid submission' STDERR: 2021-08-12T05:40:30.746980661Z STDERR: 2021-08-12T05:40:30.746986045Z Traceback (most recent call last): STDERR: 2021-08-12T05:40:30.746989025Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-08-12T05:40:30.746993557Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-08-12T05:40:30.746996803Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-08-12T05:40:30.747000553Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-08-12T05:40:30.747003453Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-08-12T05:40:30.747006933Z return self.run(fileStore) STDERR: 2021-08-12T05:40:30.747010189Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-08-12T05:40:30.747013175Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-08-12T05:40:30.747016621Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-08-12T05:40:30.747019555Z ERROR:toil.worker:Exiting the worker because of a failed job on host 72e686b9bf5d STDERR: 2021-08-12T05:40:30.747024299Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-jjr4uxlv to 0 STDERR: 2021-08-12T05:40:30.747029137Z <=========