My submission was failed when uploading the file named predictions.csv, please help :) Below is part of the log: STDERR: 2024-09-03T10:52:44.854931789Z [job upload_to_synapse.cwl] /var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tb421dt5t/tmp-outs_6xoab3$ docker \ STDERR: 2024-09-03T10:52:44.854938951Z run \ STDERR: 2024-09-03T10:52:44.854944022Z -i \ STDERR: 2024-09-03T10:52:44.854949146Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tb421dt5t/tmp-outs_6xoab3,target=/dmNCwf \ STDERR: 2024-09-03T10:52:44.854955833Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/t2n6a9yvuajqi2tg1,target=/tmp \ STDERR: 2024-09-03T10:52:44.854964060Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tmpz28b579l.tmp,target=/var/lib/cwl/stgb91642c2-2788-453c-be45-63a40f4c3544/predictions.csv,readonly \ STDERR: 2024-09-03T10:52:44.854971776Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tmp0tkjhaen.tmp,target=/var/lib/cwl/stg7b693106-7780-4a3b-94ab-106f5e3c9e50/.synapseConfig,readonly \ STDERR: 2024-09-03T10:52:44.854978797Z --workdir=/dmNCwf \ STDERR: 2024-09-03T10:52:44.854983823Z --read-only=true \ STDERR: 2024-09-03T10:52:44.854988948Z --user=0:0 \ STDERR: 2024-09-03T10:52:44.854993958Z --rm \ STDERR: 2024-09-03T10:52:44.855017884Z --env=TMPDIR=/tmp \ STDERR: 2024-09-03T10:52:44.855024609Z --env=HOME=/dmNCwf \ STDERR: 2024-09-03T10:52:44.855030146Z --cidfile=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/t2n6a9yvu5vt7qbln/20240903105112-599084.cid \ STDERR: 2024-09-03T10:52:44.855036824Z sagebionetworks/synapsepythonclient:v2.3.0 \ STDERR: 2024-09-03T10:52:44.855041987Z python3 \ STDERR: 2024-09-03T10:52:44.855046885Z upload_file.py \ STDERR: 2024-09-03T10:52:44.855051963Z -f \ STDERR: 2024-09-03T10:52:44.855057091Z /var/lib/cwl/stgb91642c2-2788-453c-be45-63a40f4c3544/predictions.csv \ STDERR: 2024-09-03T10:52:44.855062542Z -p \ STDERR: 2024-09-03T10:52:44.855067478Z syn62738036 \ STDERR: 2024-09-03T10:52:44.855072556Z -ui \ STDERR: 2024-09-03T10:52:44.855077562Z syn62737894 \ STDERR: 2024-09-03T10:52:44.855082664Z -e \ STDERR: 2024-09-03T10:52:44.855087712Z syn61913638 \ STDERR: 2024-09-03T10:52:44.855092684Z -r \ STDERR: 2024-09-03T10:52:44.855097643Z results.json \ STDERR: 2024-09-03T10:52:44.855102646Z -c \ STDERR: 2024-09-03T10:52:44.855107406Z /var/lib/cwl/stg7b693106-7780-4a3b-94ab-106f5e3c9e50/.synapseConfig STDERR: 2024-09-03T10:52:44.855113197Z INFO:cwltool:[job upload_to_synapse.cwl] /var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tb421dt5t/tmp-outs_6xoab3$ docker \ STDERR: 2024-09-03T10:52:44.855119756Z run \ STDERR: 2024-09-03T10:52:44.855124672Z -i \ STDERR: 2024-09-03T10:52:44.855129512Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tb421dt5t/tmp-outs_6xoab3,target=/dmNCwf \ STDERR: 2024-09-03T10:52:44.855137279Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/t2n6a9yvuajqi2tg1,target=/tmp \ STDERR: 2024-09-03T10:52:44.855144178Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tmpz28b579l.tmp,target=/var/lib/cwl/stgb91642c2-2788-453c-be45-63a40f4c3544/predictions.csv,readonly \ STDERR: 2024-09-03T10:52:44.855151306Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tmp0tkjhaen.tmp,target=/var/lib/cwl/stg7b693106-7780-4a3b-94ab-106f5e3c9e50/.synapseConfig,readonly \ STDERR: 2024-09-03T10:52:44.855165047Z --workdir=/dmNCwf \ STDERR: 2024-09-03T10:52:44.855170882Z --read-only=true \ STDERR: 2024-09-03T10:52:44.855175913Z --user=0:0 \ STDERR: 2024-09-03T10:52:44.855180994Z --rm \ STDERR: 2024-09-03T10:52:44.855185949Z --env=TMPDIR=/tmp \ STDERR: 2024-09-03T10:52:44.855191107Z --env=HOME=/dmNCwf \ STDERR: 2024-09-03T10:52:44.855196259Z --cidfile=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/t2n6a9yvu5vt7qbln/20240903105112-599084.cid \ STDERR: 2024-09-03T10:52:44.855202736Z sagebionetworks/synapsepythonclient:v2.3.0 \ STDERR: 2024-09-03T10:52:44.855207885Z python3 \ STDERR: 2024-09-03T10:52:44.855212926Z upload_file.py \ STDERR: 2024-09-03T10:52:44.855217898Z -f \ STDERR: 2024-09-03T10:52:44.855222945Z /var/lib/cwl/stgb91642c2-2788-453c-be45-63a40f4c3544/predictions.csv \ STDERR: 2024-09-03T10:52:44.855228226Z -p \ STDERR: 2024-09-03T10:52:44.855233637Z syn62738036 \ STDERR: 2024-09-03T10:52:44.855238563Z -ui \ STDERR: 2024-09-03T10:52:44.855243505Z syn62737894 \ STDERR: 2024-09-03T10:52:44.855248395Z -e \ STDERR: 2024-09-03T10:52:44.855253269Z syn61913638 \ STDERR: 2024-09-03T10:52:44.855258151Z -r \ STDERR: 2024-09-03T10:52:44.855262986Z results.json \ STDERR: 2024-09-03T10:52:44.855267940Z -c \ STDERR: 2024-09-03T10:52:44.855273018Z /var/lib/cwl/stg7b693106-7780-4a3b-94ab-106f5e3c9e50/.synapseConfig STDERR: 2024-09-03T10:52:44.855278208Z Traceback (most recent call last): STDERR: 2024-09-03T10:52:44.855283328Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 706, in urlopen STDERR: 2024-09-03T10:52:44.855289311Z chunked=chunked, STDERR: 2024-09-03T10:52:44.855294285Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 445, in _make_request STDERR: 2024-09-03T10:52:44.855299924Z six.raise_from(e, None) STDERR: 2024-09-03T10:52:44.855306258Z File "", line 3, in raise_from STDERR: 2024-09-03T10:52:44.855312299Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 440, in _make_request STDERR: 2024-09-03T10:52:44.855318167Z httplib_response = conn.getresponse() STDERR: 2024-09-03T10:52:44.855323315Z File "/usr/lib/python3.6/http/client.py", line 1373, in getresponse STDERR: 2024-09-03T10:52:44.855328719Z response.begin() STDERR: 2024-09-03T10:52:44.855333667Z File "/usr/lib/python3.6/http/client.py", line 311, in begin STDERR: 2024-09-03T10:52:44.855344375Z version, status, reason = self._read_status() STDERR: 2024-09-03T10:52:44.855350367Z File "/usr/lib/python3.6/http/client.py", line 272, in _read_status STDERR: 2024-09-03T10:52:44.855355846Z line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") STDERR: 2024-09-03T10:52:44.855395894Z File "/usr/lib/python3.6/socket.py", line 586, in readinto STDERR: 2024-09-03T10:52:44.855403652Z return self._sock.recv_into(b) STDERR: 2024-09-03T10:52:44.855408995Z File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into STDERR: 2024-09-03T10:52:44.855414464Z return self.read(nbytes, buffer) STDERR: 2024-09-03T10:52:44.855433505Z File "/usr/lib/python3.6/ssl.py", line 874, in read STDERR: 2024-09-03T10:52:44.855439755Z return self._sslobj.read(len, buffer) STDERR: 2024-09-03T10:52:44.855444911Z File "/usr/lib/python3.6/ssl.py", line 631, in read STDERR: 2024-09-03T10:52:44.855450087Z v = self._sslobj.read(len, buffer) STDERR: 2024-09-03T10:52:44.855455042Z ConnectionResetError: [Errno 104] Connection reset by peer STDERR: 2024-09-03T10:52:44.855460277Z STDERR: 2024-09-03T10:52:44.855465037Z During handling of the above exception, another exception occurred: STDERR: 2024-09-03T10:52:44.855470288Z STDERR: 2024-09-03T10:52:44.855474981Z Traceback (most recent call last): STDERR: 2024-09-03T10:52:44.855480033Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 449, in send STDERR: 2024-09-03T10:52:44.855485667Z timeout=timeout STDERR: 2024-09-03T10:52:44.855490498Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 756, in urlopen STDERR: 2024-09-03T10:52:44.855496235Z method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] STDERR: 2024-09-03T10:52:44.855501560Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/util/retry.py", line 531, in increment STDERR: 2024-09-03T10:52:44.855507325Z raise six.reraise(type(error), error, _stacktrace) STDERR: 2024-09-03T10:52:44.855512533Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/packages/six.py", line 734, in reraise STDERR: 2024-09-03T10:52:44.855518137Z raise value.with_traceback(tb) STDERR: 2024-09-03T10:52:44.855523054Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 706, in urlopen STDERR: 2024-09-03T10:52:44.855528681Z chunked=chunked, STDERR: 2024-09-03T10:52:44.855533648Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 445, in _make_request STDERR: 2024-09-03T10:52:44.855540251Z six.raise_from(e, None) STDERR: 2024-09-03T10:52:44.855545708Z File "", line 3, in raise_from STDERR: 2024-09-03T10:52:44.855551188Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 440, in _make_request STDERR: 2024-09-03T10:52:44.855563738Z httplib_response = conn.getresponse() STDERR: 2024-09-03T10:52:44.855569511Z File "/usr/lib/python3.6/http/client.py", line 1373, in getresponse STDERR: 2024-09-03T10:52:44.855574976Z response.begin() STDERR: 2024-09-03T10:52:44.855579901Z File "/usr/lib/python3.6/http/client.py", line 311, in begin STDERR: 2024-09-03T10:52:44.855585158Z version, status, reason = self._read_status() STDERR: 2024-09-03T10:52:44.855590375Z File "/usr/lib/python3.6/http/client.py", line 272, in _read_status STDERR: 2024-09-03T10:52:44.855595817Z line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") STDERR: 2024-09-03T10:52:44.855601137Z File "/usr/lib/python3.6/socket.py", line 586, in readinto STDERR: 2024-09-03T10:52:44.855606430Z return self._sock.recv_into(b) STDERR: 2024-09-03T10:52:44.855611320Z File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into STDERR: 2024-09-03T10:52:44.855616613Z return self.read(nbytes, buffer) STDERR: 2024-09-03T10:52:44.855621571Z File "/usr/lib/python3.6/ssl.py", line 874, in read STDERR: 2024-09-03T10:52:44.855626768Z return self._sslobj.read(len, buffer) STDERR: 2024-09-03T10:52:44.855631725Z File "/usr/lib/python3.6/ssl.py", line 631, in read STDERR: 2024-09-03T10:52:44.855637186Z v = self._sslobj.read(len, buffer) STDERR: 2024-09-03T10:52:44.855646334Z urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) STDERR: 2024-09-03T10:52:44.855652153Z STDERR: 2024-09-03T10:52:44.855657029Z During handling of the above exception, another exception occurred: STDERR: 2024-09-03T10:52:44.855662252Z STDERR: 2024-09-03T10:52:44.855667000Z Traceback (most recent call last): STDERR: 2024-09-03T10:52:44.855672069Z File "upload_file.py", line 15, in STDERR: 2024-09-03T10:52:44.855677561Z syn = synapseclient.Synapse(configPath=args.synapse_config) STDERR: 2024-09-03T10:52:44.855682906Z File "/usr/local/lib/python3.6/dist-packages/synapseclient-2.3.0-py3.6.egg/synapseclient/client.py", line 217, in __init__ STDERR: 2024-09-03T10:52:44.855688711Z self.setEndpoints(repoEndpoint, authEndpoint, fileHandleEndpoint, portalEndpoint, skip_checks) STDERR: 2024-09-03T10:52:44.855694230Z File "/usr/local/lib/python3.6/dist-packages/synapseclient-2.3.0-py3.6.egg/synapseclient/client.py", line 314, in setEndpoints STDERR: 2024-09-03T10:52:44.855699951Z headers=synapseclient.USER_AGENT) STDERR: 2024-09-03T10:52:44.855704873Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 555, in get STDERR: 2024-09-03T10:52:44.855710441Z return self.request('GET', url, **kwargs) STDERR: 2024-09-03T10:52:44.855715474Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 542, in request STDERR: 2024-09-03T10:52:44.855726760Z resp = self.send(prep, **send_kwargs) STDERR: 2024-09-03T10:52:44.855733601Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 655, in send STDERR: 2024-09-03T10:52:44.855739592Z r = adapter.send(request, **kwargs) STDERR: 2024-09-03T10:52:44.855744697Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 498, in send STDERR: 2024-09-03T10:52:44.855750266Z raise ConnectionError(err, request=request) STDERR: 2024-09-03T10:52:44.855755352Z requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) STDERR: 2024-09-03T10:52:44.855760881Z [job upload_to_synapse.cwl] Max memory used: 25MiB STDERR: 2024-09-03T10:52:44.855765983Z INFO:cwltool:[job upload_to_synapse.cwl] Max memory used: 25MiB STDERR: 2024-09-03T10:52:44.855771097Z [job upload_to_synapse.cwl] Job error:

Created by ?? ? wuhuikai
Hi, we are also encountering the "Connection reset by peer" error. Submission IDs 9749393 9749386 9749385 Submission ID 9749390 has failed after I got the email with test phase scores (?) and is now marked as INVALID. Are we doing anything wrong? @oussama.smaoui
@oussama.smaoui I retried again, it worked :)
@oussama.smaoui Please help :)

Your workflow job, (submission ID 9749308), has failed to complete. page is loading…