Dear BraTS team, Our submission (ID: 9714120) has encountered an error during scoring. Could you please help us to understand the log file and clarify where did our submission went wrong. STDERR: 2021-08-13T02:13:28.778347866Z 8c8ef9dc266c 2021-08-13 02:13:28,777 MainThread INFO cwltool: Resolved '/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/validation-phase-workflow.cwl' to 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/validation-phase-workflow.cwl' STDERR: 2021-08-13T02:13:34.004012321Z 8c8ef9dc266c 2021-08-13 02:13:34,003 MainThread WARNING cwltool: Workflow checker warning: STDERR: 2021-08-13T02:13:34.004043550Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:55:9: Source 'filepath' of type STDERR: 2021-08-13T02:13:34.004047653Z ["null", "File"] may be STDERR: 2021-08-13T02:13:34.004051960Z incompatible STDERR: 2021-08-13T02:13:34.004055038Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:75:9: with sink 'input_file' of STDERR: 2021-08-13T02:13:34.004057993Z type "File" STDERR: 2021-08-13T02:13:34.004062302Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:55:9: Source 'filepath' of type STDERR: 2021-08-13T02:13:34.004065663Z ["null", "File"] may be STDERR: 2021-08-13T02:13:34.004068990Z incompatible STDERR: 2021-08-13T02:13:34.004072986Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:135:9: with sink 'input_file' of STDERR: 2021-08-13T02:13:34.004076830Z type "File" STDERR: 2021-08-13T02:13:35.492339066Z 8c8ef9dc266c 2021-08-13 02:13:35,491 MainThread INFO toil: Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-13T02:13:36.639988222Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmp_902q6nu/worker_log.txt STDERR: 2021-08-13T02:13:37.099659399Z 8c8ef9dc266c 2021-08-13 02:13:37,099 MainThread INFO toil.leader: 0 jobs are running, 0 jobs are issued and waiting to run STDERR: 2021-08-13T02:13:37.100476345Z 8c8ef9dc266c 2021-08-13 02:13:37,100 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/set_permissions.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_set_permissions.cwl/instance-zezxn1ig with job batch system ID: 1 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:13:37.100630567Z 8c8ef9dc266c 2021-08-13 02:13:37,100 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-lagl3k1v with job batch system ID: 2 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:13:37.100765932Z 8c8ef9dc266c 2021-08-13 02:13:37,100 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-ihswbee6 with job batch system ID: 3 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:13:38.826868719Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpdkwhe_ix/worker_log.txt STDERR: 2021-08-13T02:13:38.941263608Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpyq1f6s0b/worker_log.txt STDERR: 2021-08-13T02:13:39.345363467Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpg4wg52pd/worker_log.txt STDERR: 2021-08-13T02:15:10.694159442Z 8c8ef9dc266c 2021-08-13 02:15:10,692 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-ihswbee6 STDERR: 2021-08-13T02:15:16.376292566Z 8c8ef9dc266c 2021-08-13 02:15:16,375 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/set_permissions.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_set_permissions.cwl/instance-zezxn1ig STDERR: 2021-08-13T02:15:20.619982877Z 8c8ef9dc266c 2021-08-13 02:15:20,618 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-lagl3k1v STDERR: 2021-08-13T02:15:20.620035504Z 8c8ef9dc266c 2021-08-13 02:15:20,619 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_validate.cwl/instance-z03z3dz7 with job batch system ID: 4 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:15:21.787412995Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpgyovdwqb/worker_log.txt STDERR: 2021-08-13T02:16:00.452951004Z 8c8ef9dc266c 2021-08-13 02:16:00,452 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_validate.cwl/instance-z03z3dz7 STDERR: 2021-08-13T02:16:00.454083997Z 8c8ef9dc266c 2021-08-13 02:16:00,453 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-yr1haegi with job batch system ID: 5 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:16:00.454256947Z 8c8ef9dc266c 2021-08-13 02:16:00,454 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-igoro7g8 with job batch system ID: 6 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:16:01.971321467Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpucp9ntys/worker_log.txt STDERR: 2021-08-13T02:16:02.161749092Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpnp5k3xz8/worker_log.txt STDERR: 2021-08-13T02:17:13.163674135Z 8c8ef9dc266c 2021-08-13 02:17:13,163 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-yr1haegi STDERR: 2021-08-13T02:17:18.999281290Z 8c8ef9dc266c 2021-08-13 02:17:18,998 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-igoro7g8 STDERR: 2021-08-13T02:17:18.999930524Z 8c8ef9dc266c 2021-08-13 02:17:18,999 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-5x1puizo with job batch system ID: 7 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:17:20.188076059Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmp58tdcs8k/worker_log.txt STDERR: 2021-08-13T02:17:20.594025306Z 8c8ef9dc266c 2021-08-13 02:17:20,593 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-5x1puizo STDERR: 2021-08-13T02:17:20.606366392Z 8c8ef9dc266c 2021-08-13 02:17:20,606 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_score.cwl/instance-dz19v345 with job batch system ID: 8 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:17:21.773184534Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpp9r9nypz/worker_log.txt STDERR: 2021-08-13T02:17:57.384697506Z 8c8ef9dc266c 2021-08-13 02:17:57,382 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_score.cwl/instance-dz19v345 STDERR: 2021-08-13T02:17:57.384736375Z 8c8ef9dc266c 2021-08-13 02:17:57,384 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-vzlz90l3 with job batch system ID: 9 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:17:57.384742171Z 8c8ef9dc266c 2021-08-13 02:17:57,384 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_score_email.cwl/instance-5cuv3qkp with job batch system ID: 10 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:17:59.033588606Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmp5yb23xra/worker_log.txt STDERR: 2021-08-13T02:17:59.263356366Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpumltzeto/worker_log.txt STDERR: 2021-08-13T02:19:05.063675628Z 8c8ef9dc266c 2021-08-13 02:19:05,063 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-vzlz90l3 STDERR: 2021-08-13T02:19:11.333610665Z 8c8ef9dc266c 2021-08-13 02:19:11,332 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_score_email.cwl/instance-5cuv3qkp STDERR: 2021-08-13T02:19:11.333645496Z 8c8ef9dc266c 2021-08-13 02:19:11,333 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 with job batch system ID: 11 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-13T02:19:12.555678618Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpc6pc8cx4/worker_log.txt STDERR: 2021-08-13T02:19:13.024153602Z 8c8ef9dc266c 2021-08-13 02:19:13,023 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 STDERR: 2021-08-13T02:19:13.024899196Z 8c8ef9dc266c 2021-08-13 02:19:13,024 MainThread WARNING toil.leader: The job seems to have left a log file, indicating failure: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 STDERR: 2021-08-13T02:19:13.025131611Z 8c8ef9dc266c 2021-08-13 02:19:13,024 MainThread WARNING toil.leader: Log from job kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 follows: STDERR: 2021-08-13T02:19:13.025144281Z =========> STDERR: 2021-08-13T02:19:13.025148707Z INFO:toil.worker:---TOIL WORKER OUTPUT LOG--- STDERR: 2021-08-13T02:19:13.025168752Z INFO:toil:Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-13T02:19:13.025173070Z Failed to evaluate expression: STDERR: 2021-08-13T02:19:13.025176466Z Expression evaluation error: STDERR: 2021-08-13T02:19:13.025179643Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-13T02:19:13.025182967Z script was: STDERR: 2021-08-13T02:19:13.025186209Z 01 "use strict"; STDERR: 2021-08-13T02:19:13.025189568Z 02 var inputs = { STDERR: 2021-08-13T02:19:13.025192778Z 03 "status": "INVALID", STDERR: 2021-08-13T02:19:13.025196158Z 04 "previous_annotation_finished": true, STDERR: 2021-08-13T02:19:13.025199590Z 05 "previous_email_finished": true STDERR: 2021-08-13T02:19:13.025202983Z 06 }; STDERR: 2021-08-13T02:19:13.025206187Z 07 var self = null; STDERR: 2021-08-13T02:19:13.025209521Z 08 var runtime = { STDERR: 2021-08-13T02:19:13.025212766Z 09 "cores": 1, STDERR: 2021-08-13T02:19:13.025216177Z 10 "ram": 100.0, STDERR: 2021-08-13T02:19:13.025219527Z 11 "tmpdirSize": 1024, STDERR: 2021-08-13T02:19:13.025222924Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-13T02:19:13.025226371Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/tmp_qlcxm5e", STDERR: 2021-08-13T02:19:13.025230451Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpc6pc8cx4/895437b8-4465-4f33-8d8e-1ba5443878eb/tah2om8s7/tmp-outxzytwn4h" STDERR: 2021-08-13T02:19:13.025261214Z 15 }; STDERR: 2021-08-13T02:19:13.025264588Z 16 (function(){ STDERR: 2021-08-13T02:19:13.025267831Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-13T02:19:13.025271169Z 18 throw 'invalid submission'; STDERR: 2021-08-13T02:19:13.025274413Z 19 } else { STDERR: 2021-08-13T02:19:13.025277843Z 20 return {finished: true}; STDERR: 2021-08-13T02:19:13.025281011Z 21 } STDERR: 2021-08-13T02:19:13.025284224Z 22 })() STDERR: 2021-08-13T02:19:13.025287471Z stdout was: '' STDERR: 2021-08-13T02:19:13.025299285Z stderr was: 'invalid submission' STDERR: 2021-08-13T02:19:13.025302870Z STDERR: 2021-08-13T02:19:13.025305941Z WARNING:cwltool:Failed to evaluate expression: STDERR: 2021-08-13T02:19:13.025324961Z Expression evaluation error: STDERR: 2021-08-13T02:19:13.025328955Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-13T02:19:13.025332481Z script was: STDERR: 2021-08-13T02:19:13.025335565Z 01 "use strict"; STDERR: 2021-08-13T02:19:13.025338840Z 02 var inputs = { STDERR: 2021-08-13T02:19:13.025341966Z 03 "status": "INVALID", STDERR: 2021-08-13T02:19:13.025345270Z 04 "previous_annotation_finished": true, STDERR: 2021-08-13T02:19:13.025348762Z 05 "previous_email_finished": true STDERR: 2021-08-13T02:19:13.025352088Z 06 }; STDERR: 2021-08-13T02:19:13.025355206Z 07 var self = null; STDERR: 2021-08-13T02:19:13.025358461Z 08 var runtime = { STDERR: 2021-08-13T02:19:13.025361618Z 09 "cores": 1, STDERR: 2021-08-13T02:19:13.025365052Z 10 "ram": 100.0, STDERR: 2021-08-13T02:19:13.025368369Z 11 "tmpdirSize": 1024, STDERR: 2021-08-13T02:19:13.025371728Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-13T02:19:13.025375166Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/tmp_qlcxm5e", STDERR: 2021-08-13T02:19:13.025380716Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpc6pc8cx4/895437b8-4465-4f33-8d8e-1ba5443878eb/tah2om8s7/tmp-outxzytwn4h" STDERR: 2021-08-13T02:19:13.025384972Z 15 }; STDERR: 2021-08-13T02:19:13.025402633Z 16 (function(){ STDERR: 2021-08-13T02:19:13.025406304Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-13T02:19:13.025409769Z 18 throw 'invalid submission'; STDERR: 2021-08-13T02:19:13.025413078Z 19 } else { STDERR: 2021-08-13T02:19:13.025416310Z 20 return {finished: true}; STDERR: 2021-08-13T02:19:13.025419469Z 21 } STDERR: 2021-08-13T02:19:13.025422585Z 22 })() STDERR: 2021-08-13T02:19:13.025425795Z stdout was: '' STDERR: 2021-08-13T02:19:13.025428952Z stderr was: 'invalid submission' STDERR: 2021-08-13T02:19:13.025432148Z STDERR: 2021-08-13T02:19:13.025435216Z Traceback (most recent call last): STDERR: 2021-08-13T02:19:13.025438513Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-08-13T02:19:13.025442286Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-08-13T02:19:13.025445712Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-08-13T02:19:13.025449234Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-08-13T02:19:13.025460438Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-08-13T02:19:13.025464337Z return self.run(fileStore) STDERR: 2021-08-13T02:19:13.025483791Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-08-13T02:19:13.025487494Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-08-13T02:19:13.025490741Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-08-13T02:19:13.025494050Z ERROR:toil.worker:Exiting the worker because of a failed job on host 8c8ef9dc266c STDERR: 2021-08-13T02:19:13.025497693Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 to 0 STDERR: 2021-08-13T02:19:13.025502486Z <========= STDERR: 2021-08-13T02:19:13.025561018Z 8c8ef9dc266c 2021-08-13 02:19:13,025 MainThread WARNING toil.leader: Job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 is completely failed STDERR: 2021-08-13T02:19:31.111071047Z 8c8ef9dc266c 2021-08-13 02:19:31,110 MainThread INFO toil.leader: Finished toil run with 11 failed jobs. STDERR: 2021-08-13T02:19:31.111225350Z 8c8ef9dc266c 2021-08-13 02:19:31,111 MainThread INFO toil.leader: Failed jobs at end of the run: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-lagl3k1v 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_validate.cwl/instance-z03z3dz7 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-vzlz90l3 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-igoro7g8 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_score.cwl/instance-dz19v345 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-5x1puizo 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_score_email.cwl/instance-5cuv3qkp 'CWLWorkflow' kind-CWLWorkflow/instance-yr2_gef1 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-yr1haegi 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-ihswbee6 STDERR: 2021-08-13T02:19:31.126886415Z Traceback (most recent call last): STDERR: 2021-08-13T02:19:31.126913514Z File "/usr/local/bin/toil-cwl-runner", line 8, in STDERR: 2021-08-13T02:19:31.126918451Z sys.exit(main()) STDERR: 2021-08-13T02:19:31.126922195Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 1691, in main STDERR: 2021-08-13T02:19:31.126926390Z outobj = toil.start(wf1) STDERR: 2021-08-13T02:19:31.126929945Z File "/usr/local/lib/python3.6/site-packages/toil/common.py", line 829, in start STDERR: 2021-08-13T02:19:31.126933348Z return self._runMainLoop(rootJobGraph) STDERR: 2021-08-13T02:19:31.126936611Z File "/usr/local/lib/python3.6/site-packages/toil/common.py", line 1120, in _runMainLoop STDERR: 2021-08-13T02:19:31.126940018Z jobCache=self._jobCache).run() STDERR: 2021-08-13T02:19:31.126943420Z File "/usr/local/lib/python3.6/site-packages/toil/leader.py", line 269, in run STDERR: 2021-08-13T02:19:31.126946856Z raise FailedJobsException(self.config.jobStore, self.toilState.totalFailedJobs, self.jobStore) STDERR: 2021-08-13T02:19:31.126967828Z toil.leader.FailedJobsException: The job store 'file:/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/tmpvf3fr6t7' contains 11 failed jobs: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-lagl3k1v, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_validate.cwl/instance-z03z3dz7, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-vzlz90l3, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-igoro7g8, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_score.cwl/instance-dz19v345, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-5x1puizo, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_f27c8d4f-5219-4e8f-955b-381394da3f71_brats-dream-challenge-infra-main_score_email.cwl/instance-5cuv3qkp, 'CWLWorkflow' kind-CWLWorkflow/instance-yr2_gef1, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-yr1haegi, 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-ihswbee6 STDERR: 2021-08-13T02:19:31.126992229Z Log from job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 follows: STDERR: 2021-08-13T02:19:31.126996395Z =========> STDERR: 2021-08-13T02:19:31.126999565Z INFO:toil.worker:---TOIL WORKER OUTPUT LOG--- STDERR: 2021-08-13T02:19:31.127002881Z INFO:toil:Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-13T02:19:31.127006185Z Failed to evaluate expression: STDERR: 2021-08-13T02:19:31.127010037Z Expression evaluation error: STDERR: 2021-08-13T02:19:31.127013231Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-13T02:19:31.127016426Z script was: STDERR: 2021-08-13T02:19:31.127019522Z 01 "use strict"; STDERR: 2021-08-13T02:19:31.127022768Z 02 var inputs = { STDERR: 2021-08-13T02:19:31.127025946Z 03 "status": "INVALID", STDERR: 2021-08-13T02:19:31.127029312Z 04 "previous_annotation_finished": true, STDERR: 2021-08-13T02:19:31.127032652Z 05 "previous_email_finished": true STDERR: 2021-08-13T02:19:31.127035967Z 06 }; STDERR: 2021-08-13T02:19:31.127039051Z 07 var self = null; STDERR: 2021-08-13T02:19:31.127042154Z 08 var runtime = { STDERR: 2021-08-13T02:19:31.127045385Z 09 "cores": 1, STDERR: 2021-08-13T02:19:31.127048760Z 10 "ram": 100.0, STDERR: 2021-08-13T02:19:31.127052034Z 11 "tmpdirSize": 1024, STDERR: 2021-08-13T02:19:31.127055284Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-13T02:19:31.127058662Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/tmp_qlcxm5e", STDERR: 2021-08-13T02:19:31.127065883Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpc6pc8cx4/895437b8-4465-4f33-8d8e-1ba5443878eb/tah2om8s7/tmp-outxzytwn4h" STDERR: 2021-08-13T02:19:31.127070131Z 15 }; STDERR: 2021-08-13T02:19:31.127073220Z 16 (function(){ STDERR: 2021-08-13T02:19:31.127076350Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-13T02:19:31.127079641Z 18 throw 'invalid submission'; STDERR: 2021-08-13T02:19:31.127082779Z 19 } else { STDERR: 2021-08-13T02:19:31.127086090Z 20 return {finished: true}; STDERR: 2021-08-13T02:19:31.127089280Z 21 } STDERR: 2021-08-13T02:19:31.127092407Z 22 })() STDERR: 2021-08-13T02:19:31.127095472Z stdout was: '' STDERR: 2021-08-13T02:19:31.127098634Z stderr was: 'invalid submission' STDERR: 2021-08-13T02:19:31.127101777Z STDERR: 2021-08-13T02:19:31.127104761Z WARNING:cwltool:Failed to evaluate expression: STDERR: 2021-08-13T02:19:31.127107957Z Expression evaluation error: STDERR: 2021-08-13T02:19:31.127111063Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-13T02:19:31.127114243Z script was: STDERR: 2021-08-13T02:19:31.127117297Z 01 "use strict"; STDERR: 2021-08-13T02:19:31.127120520Z 02 var inputs = { STDERR: 2021-08-13T02:19:31.127123623Z 03 "status": "INVALID", STDERR: 2021-08-13T02:19:31.127126900Z 04 "previous_annotation_finished": true, STDERR: 2021-08-13T02:19:31.127130208Z 05 "previous_email_finished": true STDERR: 2021-08-13T02:19:31.127133481Z 06 }; STDERR: 2021-08-13T02:19:31.127136537Z 07 var self = null; STDERR: 2021-08-13T02:19:31.127139593Z 08 var runtime = { STDERR: 2021-08-13T02:19:31.127142638Z 09 "cores": 1, STDERR: 2021-08-13T02:19:31.127146111Z 10 "ram": 100.0, STDERR: 2021-08-13T02:19:31.127149225Z 11 "tmpdirSize": 1024, STDERR: 2021-08-13T02:19:31.127152522Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-13T02:19:31.127155817Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/tmp_qlcxm5e", STDERR: 2021-08-13T02:19:31.127159652Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/f27c8d4f-5219-4e8f-955b-381394da3f71/node-dd019218-ec9c-4031-8cc9-5676e943edbc-16349ee66b573b0d6cff3dbc6912349f/tmpc6pc8cx4/895437b8-4465-4f33-8d8e-1ba5443878eb/tah2om8s7/tmp-outxzytwn4h" STDERR: 2021-08-13T02:19:31.127163893Z 15 }; STDERR: 2021-08-13T02:19:31.127166925Z 16 (function(){ STDERR: 2021-08-13T02:19:31.127170005Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-13T02:19:31.127173283Z 18 throw 'invalid submission'; STDERR: 2021-08-13T02:19:31.127176459Z 19 } else { STDERR: 2021-08-13T02:19:31.127182012Z 20 return {finished: true}; STDERR: 2021-08-13T02:19:31.127188033Z 21 } STDERR: 2021-08-13T02:19:31.127191144Z 22 })() STDERR: 2021-08-13T02:19:31.127194027Z stdout was: '' STDERR: 2021-08-13T02:19:31.127231078Z stderr was: 'invalid submission' STDERR: 2021-08-13T02:19:31.127234572Z STDERR: 2021-08-13T02:19:31.127238287Z Traceback (most recent call last): STDERR: 2021-08-13T02:19:31.127241590Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-08-13T02:19:31.127245537Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-08-13T02:19:31.127249002Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-08-13T02:19:31.127252360Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-08-13T02:19:31.127255182Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-08-13T02:19:31.127258678Z return self.run(fileStore) STDERR: 2021-08-13T02:19:31.127261825Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-08-13T02:19:31.127265541Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-08-13T02:19:31.127268900Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-08-13T02:19:31.127272513Z ERROR:toil.worker:Exiting the worker because of a failed job on host 8c8ef9dc266c STDERR: 2021-08-13T02:19:31.127275978Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-7oondvf3 to 0 STDERR: 2021-08-13T02:19:31.127280791Z <=========