I am getting an invalid submission on my docker model, even though the log file shows it is successful. Moreover my training stage and inferring stages also show that they have been completed. If someone can help me debug the problem it would be deeply appreciated. I have posted the log file text below. Also to be noted, the logs file of the docker model is similar to the logs file of my other successful models.
Thanks.
Logs of the model
STDERR: 2020-07-30T15:01:16.763047803Z b5aac34354c4 2020-07-30 15:01:16,762 MainThread INFO cwltool: Resolved '/var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/main_workflow.cwl' to 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/main_workflow.cwl'
STDERR: 2020-07-30T15:01:25.766422552Z b5aac34354c4 2020-07-30 15:01:25,766 MainThread WARNING cwltool: Workflow checker warning:
STDERR: 2020-07-30T15:01:25.766439062Z covid19-challenge-master/infrastructure/main_workflow.cwl:77:9: Source 'docker_repository' of type
STDERR: 2020-07-30T15:01:25.766442702Z ["null", "string"] may be
STDERR: 2020-07-30T15:01:25.766446192Z incompatible
STDERR: 2020-07-30T15:01:25.766449132Z covid19-challenge-master/infrastructure/main_workflow.cwl:196:9: with sink 'docker_repository'
STDERR: 2020-07-30T15:01:25.766452012Z of type "string"
STDERR: 2020-07-30T15:01:25.766454982Z covid19-challenge-master/infrastructure/main_workflow.cwl:78:9: Source 'docker_digest' of type
STDERR: 2020-07-30T15:01:25.766457822Z ["null", "string"] may be
STDERR: 2020-07-30T15:01:25.766460792Z incompatible
STDERR: 2020-07-30T15:01:25.766463592Z covid19-challenge-master/infrastructure/main_workflow.cwl:198:9: with sink 'docker_digest' of
STDERR: 2020-07-30T15:01:25.766466402Z type "string"
STDERR: 2020-07-30T15:01:25.766469302Z covid19-challenge-master/infrastructure/main_workflow.cwl:77:9: Source 'docker_repository' of type
STDERR: 2020-07-30T15:01:25.766472222Z ["null", "string"] may be
STDERR: 2020-07-30T15:01:25.766475132Z incompatible
STDERR: 2020-07-30T15:01:25.766477972Z covid19-challenge-master/infrastructure/main_workflow.cwl:229:9: with sink 'docker_repository'
STDERR: 2020-07-30T15:01:25.766480792Z of type "string"
STDERR: 2020-07-30T15:01:25.766483742Z covid19-challenge-master/infrastructure/main_workflow.cwl:78:9: Source 'docker_digest' of type
STDERR: 2020-07-30T15:01:25.766486602Z ["null", "string"] may be
STDERR: 2020-07-30T15:01:25.766489572Z incompatible
STDERR: 2020-07-30T15:01:25.766492412Z covid19-challenge-master/infrastructure/main_workflow.cwl:231:9: with sink 'docker_digest' of
STDERR: 2020-07-30T15:01:25.766495252Z type "string"
STDERR: 2020-07-30T15:01:25.766498122Z covid19-challenge-master/infrastructure/main_workflow.cwl:239:9: 'status' is not an input parameter
STDERR: 2020-07-30T15:01:25.766501032Z of
STDERR: 2020-07-30T15:01:25.766514672Z file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_infer_docker.cwl,
STDERR: 2020-07-30T15:01:25.766517742Z expected docker_repository,
STDERR: 2020-07-30T15:01:25.766520292Z docker_digest, submissionid,
STDERR: 2020-07-30T15:01:25.766522832Z docker_registry,
STDERR: 2020-07-30T15:01:25.766525322Z docker_authentication, parentid,
STDERR: 2020-07-30T15:01:25.766527862Z synapse_config, model, input_dir,
STDERR: 2020-07-30T15:01:25.766530402Z docker_script, quota
STDERR: 2020-07-30T15:01:27.134044428Z b5aac34354c4 2020-07-30 15:01:27,133 MainThread INFO toil: Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe.
STDERR: 2020-07-30T15:01:27.969651157Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp1ksyuo_r/worker_log.txt
STDERR: 2020-07-30T15:01:28.367585680Z b5aac34354c4 2020-07-30 15:01:28,367 MainThread INFO toil.leader: 0 jobs are running, 0 jobs are issued and waiting to run
STDERR: 2020-07-30T15:01:28.369745075Z b5aac34354c4 2020-07-30 15:01:28,369 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/create_adminsynid_annotation.cwl' echo kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_create_adminsynid_annotation.cwl/instance-ihd67j8b with job batch system ID: 1 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:28.369947096Z b5aac34354c4 2020-07-30 15:01:28,369 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_submission.cwl/instance-i98ecdaz with job batch system ID: 2 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:28.370141956Z b5aac34354c4 2020-07-30 15:01:28,369 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_docker_config.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_docker_config.cwl/instance-nl0pack2 with job batch system ID: 3 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:28.370356757Z b5aac34354c4 2020-07-30 15:01:28,370 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/set_permissions.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_set_permissions.cwl/instance-pu4g7_te with job batch system ID: 4 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:29.292824266Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp8qau1crp/worker_log.txt
STDERR: 2020-07-30T15:01:29.299830504Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp5fy8i40_/worker_log.txt
STDERR: 2020-07-30T15:01:29.312617898Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpsvfhbncu/worker_log.txt
STDERR: 2020-07-30T15:01:29.313245699Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp_lzu13dm/worker_log.txt
STDERR: 2020-07-30T15:01:29.479236898Z b5aac34354c4 2020-07-30 15:01:29,477 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/create_adminsynid_annotation.cwl' echo kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_create_adminsynid_annotation.cwl/instance-ihd67j8b
STDERR: 2020-07-30T15:01:29.479954500Z b5aac34354c4 2020-07-30 15:01:29,479 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-0xzbkmeg with job batch system ID: 5 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:30.350408681Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpc0h0_aq2/worker_log.txt
STDERR: 2020-07-30T15:01:32.522444203Z b5aac34354c4 2020-07-30 15:01:32,521 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/set_permissions.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_set_permissions.cwl/instance-pu4g7_te
STDERR: 2020-07-30T15:01:32.583765115Z b5aac34354c4 2020-07-30 15:01:32,583 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_docker_config.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_docker_config.cwl/instance-nl0pack2
STDERR: 2020-07-30T15:01:33.171966490Z b5aac34354c4 2020-07-30 15:01:33,171 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_submission.cwl/instance-i98ecdaz
STDERR: 2020-07-30T15:01:33.172666362Z b5aac34354c4 2020-07-30 15:01:33,172 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/get_dataset.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_get_dataset.cwl/instance-qu1ovfj0 with job batch system ID: 6 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:33.172778312Z b5aac34354c4 2020-07-30 15:01:33,172 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/validate_docker.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_validate_docker.cwl/instance-pgpjbi_f with job batch system ID: 7 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:33.172931033Z b5aac34354c4 2020-07-30 15:01:33,172 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/get_evaluation_id.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_get_evaluation_id.cwl/instance-hxai0l6a with job batch system ID: 8 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:34.049598150Z b5aac34354c4 2020-07-30 15:01:34,049 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-0xzbkmeg
STDERR: 2020-07-30T15:01:34.050123152Z b5aac34354c4 2020-07-30 15:01:34,049 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/create_adminsynid_annotation.cwl' echo kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_create_adminsynid_annotation.cwl/instance-ihd67j8b with job batch system ID: 9 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:34.084550952Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpjrev_zob/worker_log.txt
STDERR: 2020-07-30T15:01:34.087123959Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp2uwxbl8p/worker_log.txt
STDERR: 2020-07-30T15:01:34.107048932Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpvpvffesq/worker_log.txt
STDERR: 2020-07-30T15:01:34.963496827Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp6fkgpf41/worker_log.txt
STDERR: 2020-07-30T15:01:35.068885145Z b5aac34354c4 2020-07-30 15:01:35,068 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/create_adminsynid_annotation.cwl' echo kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_create_adminsynid_annotation.cwl/instance-ihd67j8b
STDERR: 2020-07-30T15:01:37.376695206Z b5aac34354c4 2020-07-30 15:01:37,376 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/validate_docker.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_validate_docker.cwl/instance-pgpjbi_f
STDERR: 2020-07-30T15:01:37.377358688Z b5aac34354c4 2020-07-30 15:01:37,377 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-4ayx8zbo with job batch system ID: 10 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:37.377486768Z b5aac34354c4 2020-07-30 15:01:37,377 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-see9h8ft with job batch system ID: 11 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:37.933448378Z b5aac34354c4 2020-07-30 15:01:37,932 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/get_dataset.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_get_dataset.cwl/instance-qu1ovfj0
STDERR: 2020-07-30T15:01:37.934115300Z b5aac34354c4 2020-07-30 15:01:37,933 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-b5af_29b with job batch system ID: 12 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:37.934168590Z b5aac34354c4 2020-07-30 15:01:37,934 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/dockstore-tool-synapse/v0.2/cwl/synapse-get-tool.cwl' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_dockstore-tool-synapse_v0.2_cwl_synapse-get-tool.cwl/instance-ab745qiy with job batch system ID: 13 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:37.954429244Z b5aac34354c4 2020-07-30 15:01:37,953 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/get_evaluation_id.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_get_evaluation_id.cwl/instance-hxai0l6a
STDERR: 2020-07-30T15:01:38.272709615Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp2c62wvjg/worker_log.txt
STDERR: 2020-07-30T15:01:38.285130178Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpacg8w90_/worker_log.txt
STDERR: 2020-07-30T15:01:38.874621026Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpo3zrorv0/worker_log.txt
STDERR: 2020-07-30T15:01:38.901049586Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpw_c2e5kk/worker_log.txt
STDERR: 2020-07-30T15:01:41.053099945Z b5aac34354c4 2020-07-30 15:01:41,052 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/dockstore-tool-synapse/v0.2/cwl/synapse-get-tool.cwl' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_dockstore-tool-synapse_v0.2_cwl_synapse-get-tool.cwl/instance-ab745qiy
STDERR: 2020-07-30T15:01:41.488961088Z b5aac34354c4 2020-07-30 15:01:41,488 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-4ayx8zbo
STDERR: 2020-07-30T15:01:42.591577723Z b5aac34354c4 2020-07-30 15:01:42,591 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-see9h8ft
STDERR: 2020-07-30T15:01:42.591928024Z b5aac34354c4 2020-07-30 15:01:42,591 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-2k1x0_1l with job batch system ID: 14 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:43.399642739Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmprnbnag3u/worker_log.txt
STDERR: 2020-07-30T15:01:44.686375450Z b5aac34354c4 2020-07-30 15:01:44,685 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-2k1x0_1l
STDERR: 2020-07-30T15:01:44.686957102Z b5aac34354c4 2020-07-30 15:01:44,686 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-kli_if_j with job batch system ID: 15 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:01:45.157129885Z b5aac34354c4 2020-07-30 15:01:45,156 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-b5af_29b
STDERR: 2020-07-30T15:01:45.492968933Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp__m8jcu0/worker_log.txt
STDERR: 2020-07-30T15:07:19.863507918Z b5aac34354c4 2020-07-30 15:07:19,863 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-kli_if_j
STDERR: 2020-07-30T15:07:19.864061490Z b5aac34354c4 2020-07-30 15:07:19,863 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_infer_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_run_synthetic_infer_docker.cwl/instance-gk4rmi3g with job batch system ID: 16 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:07:20.710963849Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp_qt_t_8n/worker_log.txt
STDERR: 2020-07-30T15:08:49.160150751Z b5aac34354c4 2020-07-30 15:08:49,159 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_infer_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_run_synthetic_infer_docker.cwl/instance-gk4rmi3g
STDERR: 2020-07-30T15:08:49.160686022Z b5aac34354c4 2020-07-30 15:08:49,160 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_validate.cwl/instance-k78b0th7 with job batch system ID: 17 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:08:49.160798132Z b5aac34354c4 2020-07-30 15:08:49,160 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/upload_to_synapse.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_upload_to_synapse.cwl/instance-zdbmnlki with job batch system ID: 18 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:08:50.027553614Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp2vlz56td/worker_log.txt
STDERR: 2020-07-30T15:08:50.027575194Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpkg3z78_z/worker_log.txt
STDERR: 2020-07-30T15:08:52.768566193Z b5aac34354c4 2020-07-30 15:08:52,768 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_validate.cwl/instance-k78b0th7
STDERR: 2020-07-30T15:08:52.769179015Z b5aac34354c4 2020-07-30 15:08:52,768 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-4tzge_1r with job batch system ID: 19 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:08:52.769288615Z b5aac34354c4 2020-07-30 15:08:52,769 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-w6wtaav1 with job batch system ID: 20 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:08:53.597814626Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp7ico7osv/worker_log.txt
STDERR: 2020-07-30T15:08:53.643971388Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp2m3hm4on/worker_log.txt
STDERR: 2020-07-30T15:08:55.690294799Z b5aac34354c4 2020-07-30 15:08:55,689 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/upload_to_synapse.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_upload_to_synapse.cwl/instance-zdbmnlki
STDERR: 2020-07-30T15:08:55.690674400Z b5aac34354c4 2020-07-30 15:08:55,690 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-b4ci1_gp with job batch system ID: 21 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:08:56.568231710Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpwv1jgbn0/worker_log.txt
STDERR: 2020-07-30T15:08:56.913746094Z b5aac34354c4 2020-07-30 15:08:56,913 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-w6wtaav1
STDERR: 2020-07-30T15:08:57.653797161Z b5aac34354c4 2020-07-30 15:08:57,653 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-4tzge_1r
STDERR: 2020-07-30T15:08:57.654254233Z b5aac34354c4 2020-07-30 15:08:57,654 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-7i8ec9gg with job batch system ID: 22 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:08:58.492820340Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp3qlnqcrt/worker_log.txt
STDERR: 2020-07-30T15:08:59.807127526Z b5aac34354c4 2020-07-30 15:08:59,806 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-7i8ec9gg
STDERR: 2020-07-30T15:08:59.807599677Z b5aac34354c4 2020-07-30 15:08:59,807 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/create_submission_file.cwl' python3 create.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_create_submission_file.cwl/instance-nout5xs7 with job batch system ID: 23 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:00.107832991Z b5aac34354c4 2020-07-30 15:09:00,107 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-b4ci1_gp
STDERR: 2020-07-30T15:09:00.108668243Z b5aac34354c4 2020-07-30 15:09:00,108 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/upload_to_synapse.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_upload_to_synapse.cwl/instance-zdbmnlki with job batch system ID: 24 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:00.649403233Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp0m0u8184/worker_log.txt
STDERR: 2020-07-30T15:09:01.009946246Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmprg4zyeqp/worker_log.txt
STDERR: 2020-07-30T15:09:01.144971083Z b5aac34354c4 2020-07-30 15:09:01,144 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/upload_to_synapse.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_upload_to_synapse.cwl/instance-zdbmnlki
STDERR: 2020-07-30T15:09:01.864258466Z b5aac34354c4 2020-07-30 15:09:01,863 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/create_submission_file.cwl' python3 create.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_create_submission_file.cwl/instance-nout5xs7
STDERR: 2020-07-30T15:09:01.864671477Z b5aac34354c4 2020-07-30 15:09:01,864 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/upload_to_synapse.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_upload_to_synapse.cwl/instance-bnvj935e with job batch system ID: 25 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:02.713033100Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp9u8ytu2n/worker_log.txt
STDERR: 2020-07-30T15:09:06.651198434Z b5aac34354c4 2020-07-30 15:09:06,650 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/upload_to_synapse.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_upload_to_synapse.cwl/instance-bnvj935e
STDERR: 2020-07-30T15:09:07.458705769Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpjkcvv8gq/worker_log.txt
STDERR: 2020-07-30T15:09:07.603338612Z b5aac34354c4 2020-07-30 15:09:07,602 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/submit_to_challenge.cwl' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_submit_to_challenge.cwl/instance-hxw14t66 with job batch system ID: 27 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:08.423092379Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpwc_elown/worker_log.txt
STDERR: 2020-07-30T15:09:10.395837328Z b5aac34354c4 2020-07-30 15:09:10,395 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/submit_to_challenge.cwl' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_submit_to_challenge.cwl/instance-hxw14t66
STDERR: 2020-07-30T15:09:11.243844510Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpvueztq20/worker_log.txt
STDERR: 2020-07-30T15:09:12.201233872Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpapz44qwy/worker_log.txt
STDERR: 2020-07-30T15:09:12.310292870Z b5aac34354c4 2020-07-30 15:09:12,309 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/get_evaluation_id.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_get_evaluation_id.cwl/instance-hxai0l6a with job batch system ID: 30 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:12.310309930Z b5aac34354c4 2020-07-30 15:09:12,310 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/upload_to_synapse.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_upload_to_synapse.cwl/instance-bnvj935e with job batch system ID: 31 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:13.148741408Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpff2ovn7o/worker_log.txt
STDERR: 2020-07-30T15:09:13.175583099Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpwf4qn163/worker_log.txt
STDERR: 2020-07-30T15:09:13.274265860Z b5aac34354c4 2020-07-30 15:09:13,273 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/upload_to_synapse.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_upload_to_synapse.cwl/instance-bnvj935e
STDERR: 2020-07-30T15:09:13.274633511Z b5aac34354c4 2020-07-30 15:09:13,274 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/create_submission_file.cwl' python3 create.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_create_submission_file.cwl/instance-nout5xs7 with job batch system ID: 32 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:13.290167002Z b5aac34354c4 2020-07-30 15:09:13,289 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/get_evaluation_id.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_get_evaluation_id.cwl/instance-hxai0l6a
STDERR: 2020-07-30T15:09:14.089065125Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpxylbpuc4/worker_log.txt
STDERR: 2020-07-30T15:09:14.192865680Z b5aac34354c4 2020-07-30 15:09:14,192 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/create_submission_file.cwl' python3 create.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_create_submission_file.cwl/instance-nout5xs7
STDERR: 2020-07-30T15:09:14.193244431Z b5aac34354c4 2020-07-30 15:09:14,193 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-7i8ec9gg with job batch system ID: 33 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:15.022618144Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpgrix3r2m/worker_log.txt
STDERR: 2020-07-30T15:09:15.141388818Z b5aac34354c4 2020-07-30 15:09:15,140 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-7i8ec9gg
STDERR: 2020-07-30T15:09:15.141788759Z b5aac34354c4 2020-07-30 15:09:15,141 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-w6wtaav1 with job batch system ID: 34 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:15.141976820Z b5aac34354c4 2020-07-30 15:09:15,141 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-4tzge_1r with job batch system ID: 35 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:16.025125655Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpf9hnkpso/worker_log.txt
STDERR: 2020-07-30T15:09:16.025901827Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpti10sn6l/worker_log.txt
STDERR: 2020-07-30T15:09:16.137299432Z b5aac34354c4 2020-07-30 15:09:16,136 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-w6wtaav1
STDERR: 2020-07-30T15:09:16.147151698Z b5aac34354c4 2020-07-30 15:09:16,146 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-4tzge_1r
STDERR: 2020-07-30T15:09:16.147456819Z b5aac34354c4 2020-07-30 15:09:16,147 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_validate.cwl/instance-k78b0th7 with job batch system ID: 36 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:16.959093145Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp_tslc0zh/worker_log.txt
STDERR: 2020-07-30T15:09:17.063196140Z b5aac34354c4 2020-07-30 15:09:17,062 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_validate.cwl/instance-k78b0th7
STDERR: 2020-07-30T15:09:17.063595191Z b5aac34354c4 2020-07-30 15:09:17,063 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_infer_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_run_synthetic_infer_docker.cwl/instance-gk4rmi3g with job batch system ID: 37 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:17.063736792Z b5aac34354c4 2020-07-30 15:09:17,063 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/dockstore-tool-synapse/v0.2/cwl/synapse-get-tool.cwl' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_dockstore-tool-synapse_v0.2_cwl_synapse-get-tool.cwl/instance-ab745qiy with job batch system ID: 38 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:17.895976622Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp_hte4jdu/worker_log.txt
STDERR: 2020-07-30T15:09:17.914909552Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp0mfud1dg/worker_log.txt
STDERR: 2020-07-30T15:09:18.026335907Z b5aac34354c4 2020-07-30 15:09:18,025 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_infer_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_run_synthetic_infer_docker.cwl/instance-gk4rmi3g
STDERR: 2020-07-30T15:09:18.026720298Z b5aac34354c4 2020-07-30 15:09:18,026 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-kli_if_j with job batch system ID: 39 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:18.041241406Z b5aac34354c4 2020-07-30 15:09:18,040 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/dockstore-tool-synapse/v0.2/cwl/synapse-get-tool.cwl' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_dockstore-tool-synapse_v0.2_cwl_synapse-get-tool.cwl/instance-ab745qiy
STDERR: 2020-07-30T15:09:18.834174943Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpx691cv0t/worker_log.txt
STDERR: 2020-07-30T15:09:18.941782728Z b5aac34354c4 2020-07-30 15:09:18,941 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/run_synthetic_training_docker.cwl' python kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_run_synthetic_training_docker.cwl/instance-kli_if_j
STDERR: 2020-07-30T15:09:18.942148069Z b5aac34354c4 2020-07-30 15:09:18,941 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_docker_config.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_docker_config.cwl/instance-nl0pack2 with job batch system ID: 40 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:18.942288779Z b5aac34354c4 2020-07-30 15:09:18,942 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-2k1x0_1l with job batch system ID: 41 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:18.942425629Z b5aac34354c4 2020-07-30 15:09:18,942 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/get_dataset.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_get_dataset.cwl/instance-qu1ovfj0 with job batch system ID: 42 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:19.827236159Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpdosuqrmz/worker_log.txt
STDERR: 2020-07-30T15:09:19.836857465Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp0jozmphq/worker_log.txt
STDERR: 2020-07-30T15:09:19.854953042Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpw353hs29/worker_log.txt
STDERR: 2020-07-30T15:09:19.951256517Z b5aac34354c4 2020-07-30 15:09:19,950 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_docker_config.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_docker_config.cwl/instance-nl0pack2
STDERR: 2020-07-30T15:09:19.961625105Z b5aac34354c4 2020-07-30 15:09:19,960 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_check_status.cwl/instance-2k1x0_1l
STDERR: 2020-07-30T15:09:19.961652655Z b5aac34354c4 2020-07-30 15:09:19,961 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-4ayx8zbo with job batch system ID: 43 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:19.961657855Z b5aac34354c4 2020-07-30 15:09:19,961 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-see9h8ft with job batch system ID: 44 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:19.981442067Z b5aac34354c4 2020-07-30 15:09:19,981 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/get_dataset.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_get_dataset.cwl/instance-qu1ovfj0
STDERR: 2020-07-30T15:09:20.811694883Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp17n9m604/worker_log.txt
STDERR: 2020-07-30T15:09:20.847782598Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp0ybx5p2s/worker_log.txt
STDERR: 2020-07-30T15:09:20.955398293Z b5aac34354c4 2020-07-30 15:09:20,955 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_validate_email.cwl/instance-4ayx8zbo
STDERR: 2020-07-30T15:09:20.965100148Z b5aac34354c4 2020-07-30 15:09:20,964 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_annotate_submission.cwl/instance-see9h8ft
STDERR: 2020-07-30T15:09:20.965404079Z b5aac34354c4 2020-07-30 15:09:20,965 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/validate_docker.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_validate_docker.cwl/instance-pgpjbi_f with job batch system ID: 45 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:21.783515482Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp01nb61o9/worker_log.txt
STDERR: 2020-07-30T15:09:21.893439093Z b5aac34354c4 2020-07-30 15:09:21,893 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/covid19-challenge-master/infrastructure/validate_docker.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_d43d1c64-0603-401d-85ee-e3f637d36d00_covid19-challenge-master_infrastructure_validate_docker.cwl/instance-pgpjbi_f
STDERR: 2020-07-30T15:09:21.893786464Z b5aac34354c4 2020-07-30 15:09:21,893 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_submission.cwl/instance-i98ecdaz with job batch system ID: 46 and cores: 1, disk: 1.0 G, and memory: 100.0 M
STDERR: 2020-07-30T15:09:22.709420871Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmp9md2hxtb/worker_log.txt
STDERR: 2020-07-30T15:09:22.821663208Z b5aac34354c4 2020-07-30 15:09:22,821 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v2.7/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v2.7_get_submission.cwl/instance-i98ecdaz
STDERR: 2020-07-30T15:09:23.637816386Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmph21llyvx/worker_log.txt
STDERR: 2020-07-30T15:09:24.581524362Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/node-d833f77c-6eed-4811-8f36-240ee811f5c5-79372d8564c8545262975c46157f1482/tmpe8c5m3_y/worker_log.txt
STDERR: 2020-07-30T15:09:26.889671040Z b5aac34354c4 2020-07-30 15:09:26,889 MainThread INFO toil.leader: Finished toil run successfully.
STDERR: 2020-07-30T15:09:26.908544039Z b5aac34354c4 2020-07-30 15:09:26,907 MainThread INFO toil.common: Successfully deleted the job store: FileJobStore(/var/lib/docker/volumes/workflow_orchestrator_shared/_data/d43d1c64-0603-401d-85ee-e3f637d36d00/tmpeml72og4)
STDOUT: 2020-07-30T15:09:27.204724863Z {}
Created by atasever Hi @atasever,
So I saw you had a more recent submission 9706825 that was invalid. I'm emailing you the error logs that occurred during the training phase of your submission.
Thanks,
@trberg
Hi Tim @trberg ,
I am once again facing the same problems as before, the models with ID 9706687 and 9706690 show that they working and running successfully in the logs but it is getting rejected on the UW dataset. If you could kindly look into it I would be deeply grateful.
Thank you very much,
@atasever
Hi Tim @trberg ,
Thank you so much for the help and support, I deeply appreciate the quick feedback and it makes the competition much more motivational for the participants. The model is now working and I got my results. The extended time would greatly help in making better models as the time limit was a big limiting factor. Once again deeply appreciate the help and support.
Thank you very much,
@atasever Hi @atasever,
So looking at the logs, it looks like your model went over the 20 minute time limit. We decided, based on this and other instances, that we'll be increasing the time limit for training to 1 hour. I've started the process to re-run your previously rejected submission. Hopefully that 1 hour time limit will be enough.
Thank you,
Tim Hi @trberg ,
Thank you for the prompt response. The submission id model that failed was 9706293, I then submitted a different model to make sure that it is the model giving an error not a problem in my docker container. If you look at submission id 9706342 which is again the same model I faced the same error. I still have not gotten this resolved.
Thank you once again,
@atasever Hi @atasever,
Were you able to get this resolved? It looks like you had a successful submission after the initial rejection.
Thank you,
@trberg
Drop files to upload
Model rejected even when successful in logs page is loading…