Hi Matte,
I was wondering if there is a way to download all of the BAM files at the same time given that downloading them one-by-one is really time consuming as there are hundreds of files.
Best,
Abolfazl
Created by Abolfazl Doostparast a.doostparast Hi Larsson,
Thanks for this brilliant tip!! Hi Rodrigo:
I see, you are right. For uploads we parallelize and have 10 concurrent threads pushing data but for download we do not. This is an open issue as we would expect a 2-5x speedup during download (depending on connection).
A cheap trick I have used similar to the model you describe with the while loop is to use xargs to parallelize the download. The parallelization is -P4 which performs 4 concurrent downloads:
```
cat ~/LIST_CONTAINING_SYN_IDS_TO_DOWNLOAD | xargs -P4 -n1 -I% synapse get %
``` Hi Larsson,
I believe the question was more related to the fact that the synapse utility does not allow concomitant download of multiple files. @rodrigo.duarte and @a.doostparast
If the files are accessible by a query, i.e. if you can run a query to limit the list in syn11638850 that you want you can use the command line client:
```
synapse get -q "select id from syn11638850 where <...>"
```
to do bulk downloading . For example to download every RNASeq bam file from the cmc project you would do:
```
synapse get -q "SELECT * FROM syn11638850 where fileFormat='bam' and assay='rnaSeq'"
```
If you want to bulk download everything in a folder you can also do recursive download
```
synapse get -r syn123
```
where syn123 is a folder or project.
@rodrigo.duarte - thanks! Thanks Rodrigo.
I just obtained the IDs.
Best,
Abolfazl I don't think the synapse utility has a function to download many files in parallel unfortunately... To download these files more quickly, I extracted the syn ID for each sample within the dataset (it's possible to do this here:
https://www.synapse.org/#!Synapse:syn11638850/tables/ ). Then I split the ID names I wanted to download into 4-5 text files, and created 4-5 scripts containing "for" loops to download lists of individual IDs in parallel.. Example:
```
cat ~/LIST_CONTAINING_SYN_IDS_TO_DOWNLOAD | while read line
do
echo "Downloading $line..."
synapse -u USERNAME -p PASSWORD get $line
done
```
Not very fancy but it works.
Drop files to upload
Downloading all of the BAM files as a single batch page is loading…