I am trying to download via CLI, and using a recently updated client, the downloaded files keep erasing themselves and starting over once they get to 100MB. It will do this with six files simultaneously, and the "Downloaded" status bar will reverse, and the downloaded size will get smaller, not bigger(?!). Eventually, after about an hour, it will error out with:
```
ValueError: File download failed during sync
```
synapseclient 2.2.0 from bioconda
Created by Robert Butler rbutleriii The above referenced fix improving handling of out of disk space errors is now included in the latest release (2.2.2) of the Synapse Python client.
It can be installed e.g.:
```
pip install --upgrade synapseclient
```
@rbutleriii and @lukais.iohan a release candidate of the next Python client includes a fix that should cause the client to abort a download more gracefully when the disk space is exhausted. If the underlying error is something else it should also do a better job of printing what the cause is when exiting (as opposed to just "File download failed during sync").
The release candidate can be installed e.g.
```
pip3 install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple "synapseclient>=2.2.2"
```
I'd appreciate it if you had a chance to confirm if it works better for you or reports a different error.
@rbutleriii and @lukais.iohan Thanks for the report.
I have a consistent way to reproduce this but in my case it does involve running out of disk space. What happens is that each parallel download thread tries to download a file, encounters an out of disk space error, deletes the partially downloaded file. When one thread deletes its partial download it frees up space for another thread to attempt to continue and so on. The Downloaded progress indicator is reflecting the loss of progress after having deleted a partial download.
However @rbutleriii it sounds like you had multiple TB of free space so I'm not sure if the underlying cause is the same in your case.
If you have a chance I would appreciate it if you could reinstall the latest version of the synapse client ( **pip install --upgrade synapseclient** ) and run the following script (the programmatic equivalent of **synapse get -r ** in order to configure some extra debug level logging) and let me know any stack traces that may appear during the download.
```
import synapseclient
import synapseutils
import logging
logging.basicConfig()
logging.getLogger(synapseclient.client.DEFAULT_LOGGER_NAME).setLevel(logging.DEBUG)
SYNAPSE_FOLDER_ID = 'syn16780177' # replace with the Synapse ID of the folder being downloaded
DOWNLOAD_PATH = None # replace if the download is to some other path than the defualt (synapse cache)
syn = synapseclient.login()
synapseutils.syncFromSynapse(syn, SYNAPSE_FOLDER_ID, path=DOWNLOAD_PATH)
```
I think this should show the underlying error that is causing the reset file downloads in your case, whether it is a disk space issue or something else. I'll have a fix that will allow the client to behave more gracefully in the case where available disk space is exhausted and doesn't treat that as a retry-able scenario, but I want to be sure I handle the possibility of another error if the underlying cause in your cause is a different Exception. I had the same issue with version 2.20 installed from conda . I've downgranded to 1.7.5 e worked fine to me. `synapse get -r syn16780177`
There are 2+ TB on the drive, so it should be able to hold the folder.
I tried rolling back the synapse version to `1.7.5`, and it was able to download files. Though it changed the file download order and only downloaded one file at a time. When I started getting the files one at a time by synID it worked with `2.2.0`. Seems like it could be something with the multiThreaded option? @rbutleriii Can you include the command you are running from the CLI, including the synapse id of the folder you are downloading?
Also can you ensure that you have adequate disk space on the volume you are downloading to store the downloaded files?