I have been asked to download several studies:
https://www.synapse.org/#!Synapse:syn4566010
https://www.synapse.org/#!Synapse:syn5321694
https://www.synapse.org/#!Synapse:syn5842535
https://www.synapse.org/#!Synapse:syn4587609
https://www.synapse.org/#!Synapse:syn4566141
https://www.synapse.org/#!Synapse:syn4588462
I installed the CLI tool to download the files. I have kicked off the first two and they crawled along slowly. Currently they have all but halted, Making no progress on 6-7GB files over the last couple of hours. Is there any quicker way to download these studies, like and FTP site? Or is there something else slowing these downloads down to a snail's pace?
Thanks
Created by neil ihrke nihrke @nihrke, Is it possible that you are getting throttled? If you download the same file using your web-browser what throughput are you seeing?
If it helps, feel free to reach out to me via email as well larssono@synapse.org.
I tried running just one download with the debug option. The download starts out around 30MB/s, but quickly drops off.
Downloading [#######-------------]33.06% 4.2GB/12.8GB (2.0MB/s)
So far, i don't see any additonal output using the --debug option.
Ouch, that is slow! And will not work to download the all the files you want. Do you mind trying restarting the download - don't worry about loosing the completed downloads, the download will restart from where you left off. To exit the download hit Ctrl-C. Then restart the download but include the parameter --debug in your download. E.g. if you were using:
```
synapse get -r syn4566010
```
it would become:
```
synapse --debug get -r syn4566010
```
This will give a bunch of information what is happening in the background. It is worth seeing if the transfer is initially faster and then drops off. Also, if you can share what the output is when it is stalled we might be able to find a cause.
A short note - although very slow it is not quite as slow as you suggesting 2MB/s ~ 20mbps it says around 2mbps, but this current file hasnt progressed in over an hour. its been stuck on the same percentage:
Downloading [######--------------]30.69% 2.1GB/6.8GB (2.1MB/s) Thanks for the quick response. How slow? The rate is reported in the progress bar while downloading. I am in US Central region. I have two downloads running, but have noticed the slow moving downloads when only one was running. Hi @nihrke
Sorry, to hear that things are moving slowly. To get some clarity, how slow are they moving and how many concurrent downloads are you running? Also, out of curiosity where are you located? We usually see people saturating their own network bandwidth before the bandwidth of the the Amazon S3 bucket where the data is stored. The files you are downloading are stored on amazon S3 in the US east region. There are tweaks on a good GigE connection to go from ~30MB/s to ~80MB/s by changing TCP parameters if you are geographically far away from the East coast. But it sounds like you are seeing significantly slower speeds than this.