My submission was failed when uploading the file named predictions.csv, please help :)
Below is part of the log:
STDERR: 2024-09-03T10:52:44.854931789Z [job upload_to_synapse.cwl] /var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tb421dt5t/tmp-outs_6xoab3$ docker \
STDERR: 2024-09-03T10:52:44.854938951Z run \
STDERR: 2024-09-03T10:52:44.854944022Z -i \
STDERR: 2024-09-03T10:52:44.854949146Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tb421dt5t/tmp-outs_6xoab3,target=/dmNCwf \
STDERR: 2024-09-03T10:52:44.854955833Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/t2n6a9yvuajqi2tg1,target=/tmp \
STDERR: 2024-09-03T10:52:44.854964060Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tmpz28b579l.tmp,target=/var/lib/cwl/stgb91642c2-2788-453c-be45-63a40f4c3544/predictions.csv,readonly \
STDERR: 2024-09-03T10:52:44.854971776Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tmp0tkjhaen.tmp,target=/var/lib/cwl/stg7b693106-7780-4a3b-94ab-106f5e3c9e50/.synapseConfig,readonly \
STDERR: 2024-09-03T10:52:44.854978797Z --workdir=/dmNCwf \
STDERR: 2024-09-03T10:52:44.854983823Z --read-only=true \
STDERR: 2024-09-03T10:52:44.854988948Z --user=0:0 \
STDERR: 2024-09-03T10:52:44.854993958Z --rm \
STDERR: 2024-09-03T10:52:44.855017884Z --env=TMPDIR=/tmp \
STDERR: 2024-09-03T10:52:44.855024609Z --env=HOME=/dmNCwf \
STDERR: 2024-09-03T10:52:44.855030146Z --cidfile=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/t2n6a9yvu5vt7qbln/20240903105112-599084.cid \
STDERR: 2024-09-03T10:52:44.855036824Z sagebionetworks/synapsepythonclient:v2.3.0 \
STDERR: 2024-09-03T10:52:44.855041987Z python3 \
STDERR: 2024-09-03T10:52:44.855046885Z upload_file.py \
STDERR: 2024-09-03T10:52:44.855051963Z -f \
STDERR: 2024-09-03T10:52:44.855057091Z /var/lib/cwl/stgb91642c2-2788-453c-be45-63a40f4c3544/predictions.csv \
STDERR: 2024-09-03T10:52:44.855062542Z -p \
STDERR: 2024-09-03T10:52:44.855067478Z syn62738036 \
STDERR: 2024-09-03T10:52:44.855072556Z -ui \
STDERR: 2024-09-03T10:52:44.855077562Z syn62737894 \
STDERR: 2024-09-03T10:52:44.855082664Z -e \
STDERR: 2024-09-03T10:52:44.855087712Z syn61913638 \
STDERR: 2024-09-03T10:52:44.855092684Z -r \
STDERR: 2024-09-03T10:52:44.855097643Z results.json \
STDERR: 2024-09-03T10:52:44.855102646Z -c \
STDERR: 2024-09-03T10:52:44.855107406Z /var/lib/cwl/stg7b693106-7780-4a3b-94ab-106f5e3c9e50/.synapseConfig
STDERR: 2024-09-03T10:52:44.855113197Z INFO:cwltool:[job upload_to_synapse.cwl] /var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tb421dt5t/tmp-outs_6xoab3$ docker \
STDERR: 2024-09-03T10:52:44.855119756Z run \
STDERR: 2024-09-03T10:52:44.855124672Z -i \
STDERR: 2024-09-03T10:52:44.855129512Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tb421dt5t/tmp-outs_6xoab3,target=/dmNCwf \
STDERR: 2024-09-03T10:52:44.855137279Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/t2n6a9yvuajqi2tg1,target=/tmp \
STDERR: 2024-09-03T10:52:44.855144178Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tmpz28b579l.tmp,target=/var/lib/cwl/stgb91642c2-2788-453c-be45-63a40f4c3544/predictions.csv,readonly \
STDERR: 2024-09-03T10:52:44.855151306Z --mount=type=bind,source=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/tmp0tkjhaen.tmp,target=/var/lib/cwl/stg7b693106-7780-4a3b-94ab-106f5e3c9e50/.synapseConfig,readonly \
STDERR: 2024-09-03T10:52:44.855165047Z --workdir=/dmNCwf \
STDERR: 2024-09-03T10:52:44.855170882Z --read-only=true \
STDERR: 2024-09-03T10:52:44.855175913Z --user=0:0 \
STDERR: 2024-09-03T10:52:44.855180994Z --rm \
STDERR: 2024-09-03T10:52:44.855185949Z --env=TMPDIR=/tmp \
STDERR: 2024-09-03T10:52:44.855191107Z --env=HOME=/dmNCwf \
STDERR: 2024-09-03T10:52:44.855196259Z --cidfile=/var/lib/docker/volumes/workflow_orchestrator_shared/_data/5750be3a-019c-4444-a6ae-693a44f26ff8/node-1e001d79-b08d-426a-834c-ae8d2629d02a-1cc402dd0e11d5ae18db04a6de87223d/tmp5q7ukmf9/6b31ed05-f5fa-41b8-ac1e-07a4d6e18dff/t2n6a9yvu5vt7qbln/20240903105112-599084.cid \
STDERR: 2024-09-03T10:52:44.855202736Z sagebionetworks/synapsepythonclient:v2.3.0 \
STDERR: 2024-09-03T10:52:44.855207885Z python3 \
STDERR: 2024-09-03T10:52:44.855212926Z upload_file.py \
STDERR: 2024-09-03T10:52:44.855217898Z -f \
STDERR: 2024-09-03T10:52:44.855222945Z /var/lib/cwl/stgb91642c2-2788-453c-be45-63a40f4c3544/predictions.csv \
STDERR: 2024-09-03T10:52:44.855228226Z -p \
STDERR: 2024-09-03T10:52:44.855233637Z syn62738036 \
STDERR: 2024-09-03T10:52:44.855238563Z -ui \
STDERR: 2024-09-03T10:52:44.855243505Z syn62737894 \
STDERR: 2024-09-03T10:52:44.855248395Z -e \
STDERR: 2024-09-03T10:52:44.855253269Z syn61913638 \
STDERR: 2024-09-03T10:52:44.855258151Z -r \
STDERR: 2024-09-03T10:52:44.855262986Z results.json \
STDERR: 2024-09-03T10:52:44.855267940Z -c \
STDERR: 2024-09-03T10:52:44.855273018Z /var/lib/cwl/stg7b693106-7780-4a3b-94ab-106f5e3c9e50/.synapseConfig
STDERR: 2024-09-03T10:52:44.855278208Z Traceback (most recent call last):
STDERR: 2024-09-03T10:52:44.855283328Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 706, in urlopen
STDERR: 2024-09-03T10:52:44.855289311Z chunked=chunked,
STDERR: 2024-09-03T10:52:44.855294285Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 445, in _make_request
STDERR: 2024-09-03T10:52:44.855299924Z six.raise_from(e, None)
STDERR: 2024-09-03T10:52:44.855306258Z File "", line 3, in raise_from
STDERR: 2024-09-03T10:52:44.855312299Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 440, in _make_request
STDERR: 2024-09-03T10:52:44.855318167Z httplib_response = conn.getresponse()
STDERR: 2024-09-03T10:52:44.855323315Z File "/usr/lib/python3.6/http/client.py", line 1373, in getresponse
STDERR: 2024-09-03T10:52:44.855328719Z response.begin()
STDERR: 2024-09-03T10:52:44.855333667Z File "/usr/lib/python3.6/http/client.py", line 311, in begin
STDERR: 2024-09-03T10:52:44.855344375Z version, status, reason = self._read_status()
STDERR: 2024-09-03T10:52:44.855350367Z File "/usr/lib/python3.6/http/client.py", line 272, in _read_status
STDERR: 2024-09-03T10:52:44.855355846Z line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
STDERR: 2024-09-03T10:52:44.855395894Z File "/usr/lib/python3.6/socket.py", line 586, in readinto
STDERR: 2024-09-03T10:52:44.855403652Z return self._sock.recv_into(b)
STDERR: 2024-09-03T10:52:44.855408995Z File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into
STDERR: 2024-09-03T10:52:44.855414464Z return self.read(nbytes, buffer)
STDERR: 2024-09-03T10:52:44.855433505Z File "/usr/lib/python3.6/ssl.py", line 874, in read
STDERR: 2024-09-03T10:52:44.855439755Z return self._sslobj.read(len, buffer)
STDERR: 2024-09-03T10:52:44.855444911Z File "/usr/lib/python3.6/ssl.py", line 631, in read
STDERR: 2024-09-03T10:52:44.855450087Z v = self._sslobj.read(len, buffer)
STDERR: 2024-09-03T10:52:44.855455042Z ConnectionResetError: [Errno 104] Connection reset by peer
STDERR: 2024-09-03T10:52:44.855460277Z
STDERR: 2024-09-03T10:52:44.855465037Z During handling of the above exception, another exception occurred:
STDERR: 2024-09-03T10:52:44.855470288Z
STDERR: 2024-09-03T10:52:44.855474981Z Traceback (most recent call last):
STDERR: 2024-09-03T10:52:44.855480033Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 449, in send
STDERR: 2024-09-03T10:52:44.855485667Z timeout=timeout
STDERR: 2024-09-03T10:52:44.855490498Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 756, in urlopen
STDERR: 2024-09-03T10:52:44.855496235Z method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
STDERR: 2024-09-03T10:52:44.855501560Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/util/retry.py", line 531, in increment
STDERR: 2024-09-03T10:52:44.855507325Z raise six.reraise(type(error), error, _stacktrace)
STDERR: 2024-09-03T10:52:44.855512533Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/packages/six.py", line 734, in reraise
STDERR: 2024-09-03T10:52:44.855518137Z raise value.with_traceback(tb)
STDERR: 2024-09-03T10:52:44.855523054Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 706, in urlopen
STDERR: 2024-09-03T10:52:44.855528681Z chunked=chunked,
STDERR: 2024-09-03T10:52:44.855533648Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 445, in _make_request
STDERR: 2024-09-03T10:52:44.855540251Z six.raise_from(e, None)
STDERR: 2024-09-03T10:52:44.855545708Z File "", line 3, in raise_from
STDERR: 2024-09-03T10:52:44.855551188Z File "/usr/local/lib/python3.6/dist-packages/urllib3-1.26.3-py3.6.egg/urllib3/connectionpool.py", line 440, in _make_request
STDERR: 2024-09-03T10:52:44.855563738Z httplib_response = conn.getresponse()
STDERR: 2024-09-03T10:52:44.855569511Z File "/usr/lib/python3.6/http/client.py", line 1373, in getresponse
STDERR: 2024-09-03T10:52:44.855574976Z response.begin()
STDERR: 2024-09-03T10:52:44.855579901Z File "/usr/lib/python3.6/http/client.py", line 311, in begin
STDERR: 2024-09-03T10:52:44.855585158Z version, status, reason = self._read_status()
STDERR: 2024-09-03T10:52:44.855590375Z File "/usr/lib/python3.6/http/client.py", line 272, in _read_status
STDERR: 2024-09-03T10:52:44.855595817Z line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
STDERR: 2024-09-03T10:52:44.855601137Z File "/usr/lib/python3.6/socket.py", line 586, in readinto
STDERR: 2024-09-03T10:52:44.855606430Z return self._sock.recv_into(b)
STDERR: 2024-09-03T10:52:44.855611320Z File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into
STDERR: 2024-09-03T10:52:44.855616613Z return self.read(nbytes, buffer)
STDERR: 2024-09-03T10:52:44.855621571Z File "/usr/lib/python3.6/ssl.py", line 874, in read
STDERR: 2024-09-03T10:52:44.855626768Z return self._sslobj.read(len, buffer)
STDERR: 2024-09-03T10:52:44.855631725Z File "/usr/lib/python3.6/ssl.py", line 631, in read
STDERR: 2024-09-03T10:52:44.855637186Z v = self._sslobj.read(len, buffer)
STDERR: 2024-09-03T10:52:44.855646334Z urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
STDERR: 2024-09-03T10:52:44.855652153Z
STDERR: 2024-09-03T10:52:44.855657029Z During handling of the above exception, another exception occurred:
STDERR: 2024-09-03T10:52:44.855662252Z
STDERR: 2024-09-03T10:52:44.855667000Z Traceback (most recent call last):
STDERR: 2024-09-03T10:52:44.855672069Z File "upload_file.py", line 15, in
STDERR: 2024-09-03T10:52:44.855677561Z syn = synapseclient.Synapse(configPath=args.synapse_config)
STDERR: 2024-09-03T10:52:44.855682906Z File "/usr/local/lib/python3.6/dist-packages/synapseclient-2.3.0-py3.6.egg/synapseclient/client.py", line 217, in __init__
STDERR: 2024-09-03T10:52:44.855688711Z self.setEndpoints(repoEndpoint, authEndpoint, fileHandleEndpoint, portalEndpoint, skip_checks)
STDERR: 2024-09-03T10:52:44.855694230Z File "/usr/local/lib/python3.6/dist-packages/synapseclient-2.3.0-py3.6.egg/synapseclient/client.py", line 314, in setEndpoints
STDERR: 2024-09-03T10:52:44.855699951Z headers=synapseclient.USER_AGENT)
STDERR: 2024-09-03T10:52:44.855704873Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 555, in get
STDERR: 2024-09-03T10:52:44.855710441Z return self.request('GET', url, **kwargs)
STDERR: 2024-09-03T10:52:44.855715474Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 542, in request
STDERR: 2024-09-03T10:52:44.855726760Z resp = self.send(prep, **send_kwargs)
STDERR: 2024-09-03T10:52:44.855733601Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 655, in send
STDERR: 2024-09-03T10:52:44.855739592Z r = adapter.send(request, **kwargs)
STDERR: 2024-09-03T10:52:44.855744697Z File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 498, in send
STDERR: 2024-09-03T10:52:44.855750266Z raise ConnectionError(err, request=request)
STDERR: 2024-09-03T10:52:44.855755352Z requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
STDERR: 2024-09-03T10:52:44.855760881Z [job upload_to_synapse.cwl] Max memory used: 25MiB
STDERR: 2024-09-03T10:52:44.855765983Z INFO:cwltool:[job upload_to_synapse.cwl] Max memory used: 25MiB
STDERR: 2024-09-03T10:52:44.855771097Z [job upload_to_synapse.cwl] Job error:
Created by ?? ? chohotech-wuhuikai Hi, we are also encountering the "Connection reset by peer" error. Submission IDs 9749393 9749386 9749385
Submission ID 9749390 has failed after I got the email with test phase scores (?) and is now marked as INVALID.
Are we doing anything wrong? @oussama.smaoui @oussama.smaoui I retried again, it worked :) @oussama.smaoui Please help :)
Drop files to upload
Your workflow job, (submission ID 9749308), has failed to complete. page is loading…