The data is too big.How do you download it? Please~
Created by
Zhenliang Ni ZhenliangNi
@Mordokkai yes synapse get will check the destination download path and won't re-download the file it it already exists at the destination path, so resuming an interrupted download is fine. If you download the file to an alternate path then where it may have been previously downloaded and there is an entry for the file in the synapse cache (such as the entry file that was corrupted in your case) then it can additionally copy the file from that existing location rather than re-download it.
Hi @jordank, Your solution worked perfectly, thanks a lot! By the way, do you know if the command "synapse get -r ..." checks if the destination folder already contains downloaded data ? Typically if the download was interrupted, can i relaunch this command without downloading from zero again ? Thanks in advance,
@Mordokkai I was not able to reproduce an issue downloading syn20575556. From the provided stack trace (thanks!) it looks like there may be a corrupt cache map file in the synapseCache on your machine for an entity in this folder. Whenever you download a file via synapse it creates a cache map entry that tells Synapse where the entity was downloaded. This is a JSON file, but it looks like one of these files in your cache may be invalid JSON (possibly an empty file). Assuming you are running on a unix compatible system (looks like from the paths), can you try running the following command? ``` find ~/.synapseCache/ -name .cacheMap -size 0 ``` This will show any empty cacheMap files in your synapseCache which could result in the above stack trace. If one is found it can be manually deleted or deleted via e.g. the following: ``` find ~/.synapseCache/ -name .cacheMap -size 0 -delete ``` If a corrupt cacheMap entry file(s) are found and removed above, then hopefully re-downloadiing the Synapse folder will succeed. If the above does not find an empty corrupt cacheMap file, a more drastic step would be to delete your synapseCache completely, e.g. ``` rm -rf ~/.synapseCache ``` This may result in needing to re-download other files to rebuild the cache.
Dear @jordank, can you please help?
Hi all, I'm trying to download the dataset but I keep getting an error with the folder Training/Rectal_resection/1/ (id syn20575556). I downloaded separately the other subfolders with ids syn20586618, syn20587296, syn20587898, syn20588572, syn20589279, syn20591106, syn20576482, without any problems. The error message for Training/Rectal_resection/1/ is: (click on the button)
error
Downloading [####################]100.00% 2.4kB/2.4kB (2.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 421.3kB/421.3kB (961.9kB/s) raw.png Done... Downloading [####################]100.00% 372.8kB/372.8kB (1016.8kB/s) raw.png Done... Downloading [####################]100.00% 2.4kB/2.4kB (2.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 434.1kB/434.1kB (4.2MB/s) raw.png Done... Downloading [####################]100.00% 2.1kB/2.1kB (2.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 405.6kB/405.6kB (99.0MB/s) raw.png Done... Downloading [####################]100.00% 1.1kB/1.1kB (1.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 464.7kB/464.7kB (90.0MB/s) raw.png Done... Downloading [####################]100.00% 63.4MB/63.4MB (6.9MB/s) 10s_video.zip.synapse_download_42122448 Done... Downloading [####################]100.00% 255.2kB/255.2kB (695.3kB/s) raw.png Done... Downloading [####################]100.00% 103.8MB/103.8MB (3.3MB/s) 10s_video.zip.synapse_download_40879307 Done... Downloading [####################]100.00% 3.3kB/3.3kB (3.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 386.3kB/386.3kB (887.3kB/s) raw.png Done... Downloading [####################]100.00% 107.1MB/107.1MB (3.4MB/s) 10s_video.zip.synapse_download_40879287 Done... Downloading [####################]100.00% 1.1kB/1.1kB (7.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 455.5kB/455.5kB (5.0MB/s) raw.png Done... Downloading [####################]100.00% 94.3MB/94.3MB (2.5MB/s) 10s_video.zip.synapse_download_40879313 Done... Downloading [####################]100.00% 3.4kB/3.4kB (21.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 436.9kB/436.9kB (93.7MB/s) raw.png Done... Downloading [####################]100.00% 96.2MB/96.2MB (2.2MB/s) 10s_video.zip.synapse_download_42122460 Done... Downloading [####################]100.00% 481.7kB/481.7kB (84.2MB/s) raw.png Done... Downloading [####################]100.00% 91.4MB/91.4MB (3.9MB/s) 10s_video.zip.synapse_download_40879261 Done... Downloading [####################]100.00% 593.0kB/593.0kB (1.3MB/s) raw.png Done... Downloading [####################]100.00% 100.5MB/100.5MB (3.9MB/s) 10s_video.zip.synapse_download_40879269 Done... Downloading [####################]100.00% 101.1MB/101.1MB (1.7MB/s) 10s_video.zip.synapse_download_40879310 Done... Downloading [####################]100.00% 455.5kB/455.5kB (4.7MB/s) raw.png Done... Downloading [####################]100.00% 3.3kB/3.3kB (12.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 109.0MB/109.0MB (2.3MB/s) 10s_video.zip.synapse_download_40879316 Done... Downloading [####################]100.00% 423.3kB/423.3kB (4.4MB/s) raw.png Done... Downloading [####################]100.00% 1.8kB/1.8kB (6.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 503.0kB/503.0kB (5.1MB/s) raw.png Done... Downloading [####################]100.00% 113.6MB/113.6MB (3.9MB/s) 10s_video.zip.synapse_download_42122455 Done... Downloading [####################]100.00% 2.4kB/2.4kB (7.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 128.2MB/128.2MB (3.7MB/s) 10s_video.zip.synapse_download_42122454 Done... Downloading [####################]100.00% 483.0kB/483.0kB (1.3MB/s) raw.png Done... Downloading [####################]100.00% 1.1kB/1.1kB (2.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 269.4kB/269.4kB (59.2MB/s) raw.png Done... Downloading [####################]100.00% 107.0MB/107.0MB (3.9MB/s) 10s_video.zip.synapse_download_42122457 Done... Downloading [####################]100.00% 46.7MB/46.7MB (3.5MB/s) 10s_video.zip.synapse_download_40879241 Done... Downloading [####################]100.00% 1.9kB/1.9kB (4.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 445.8kB/445.8kB (815.4kB/s) raw.png Done... Downloading [####################]100.00% 4.8kB/4.8kB (20.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 99.9MB/99.9MB (3.9MB/s) 10s_video.zip.synapse_download_40879274 Done... Downloading [####################]100.00% 499.3kB/499.3kB (5.2MB/s) raw.png Done... Downloading [####################]100.00% 2.2kB/2.2kB (11.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 458.4kB/458.4kB (121.8MB/s) raw.png Done... Downloading [####################]100.00% 114.0MB/114.0MB (6.7MB/s) 10s_video.zip.synapse_download_40879250 Done... Downloading [####################]100.00% 442.4kB/442.4kB (99.8MB/s) raw.png Done... Downloading [####################]100.00% 122.9MB/122.9MB (3.7MB/s) 10s_video.zip.synapse_download_40879266 Done... Downloading [####################]100.00% 109.9MB/109.9MB (3.7MB/s) 10s_video.zip.synapse_download_40879268 Done... Downloading [####################]100.00% 1.9kB/1.9kB (8.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 478.4kB/478.4kB (1.0MB/s) raw.png Done... Downloading [####################]100.00% 2.2kB/2.2kB (8.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 408.8kB/408.8kB (4.3MB/s) raw.png Done... Downloading [####################]100.00% 97.3MB/97.3MB (5.5MB/s) 10s_video.zip.synapse_download_40879242 Done... Downloading [####################]100.00% 1.9kB/1.9kB (10.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 368.9kB/368.9kB (850.6kB/s) raw.png Done... Downloading [####################]100.00% 92.4MB/92.4MB (7.7MB/s) 10s_video.zip.synapse_download_40879217 Done... Downloading [####################]100.00% 2.9kB/2.9kB (10.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 354.7kB/354.7kB (4.0MB/s) raw.png Done... Downloading [####################]100.00% 97.9MB/97.9MB (3.3MB/s) 10s_video.zip.synapse_download_42122443 Done... Downloading [####################]100.00% 83.7MB/83.7MB (3.6MB/s) 10s_video.zip.synapse_download_40879199 Done... Downloading [####################]100.00% 2.4kB/2.4kB (13.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 415.2kB/415.2kB (532.3kB/s) raw.png Done... Downloading [####################]100.00% 483.1kB/483.1kB (792.8kB/s) raw.png Done... Downloading [####################]100.00% 104.8MB/104.8MB (6.3MB/s) 10s_video.zip.synapse_download_40879198 Done... Downloading [####################]100.00% 425.5kB/425.5kB (815.9kB/s) raw.png Done... Downloading [####################]100.00% 98.4MB/98.4MB (2.6MB/s) 10s_video.zip.synapse_download_40879229 Done... Downloading [####################]100.00% 100.2MB/100.2MB (2.9MB/s) 10s_video.zip.synapse_download_40879234 Done... Downloading [####################]100.00% 1.7kB/1.7kB (7.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 329.4kB/329.4kB (940.2kB/s) raw.png Done... Downloading [####################]100.00% 1.2kB/1.2kB (4.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 435.0kB/435.0kB (456.5kB/s) raw.png Done... Downloading [####################]100.00% 89.5MB/89.5MB (3.2MB/s) 10s_video.zip.synapse_download_40879188 Done... Downloading [####################]100.00% 3.0kB/3.0kB (13.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 334.6kB/334.6kB (552.0kB/s) raw.png Done... Downloading [####################]100.00% 85.7MB/85.7MB (8.9MB/s) 10s_video.zip.synapse_download_40879175 Done... Downloading [####################]100.00% 2.3kB/2.3kB (11.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 360.8kB/360.8kB (832.5kB/s) raw.png Done... Downloading [####################]100.00% 65.2kB/65.2kB (99.4MB/s) 10s_video.zip Done... Downloading [####################]100.00% 2.1kB/2.1kB (9.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 77.9kB/77.9kB (910.4kB/s) raw.png Done... Downloading [####################]100.00% 111.0MB/111.0MB (3.9MB/s) 10s_video.zip.synapse_download_42122434 Done... Downloading [####################]100.00% 68.0MB/68.0MB (2.8MB/s) 10s_video.zip.synapse_download_40879185 Done... Downloading [####################]100.00% 2.4kB/2.4kB (12.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 286.9kB/286.9kB (827.4kB/s) raw.png Done... Downloading [####################]100.00% 2.6kB/2.6kB (13.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 301.4kB/301.4kB (3.4MB/s) raw.png Done... Downloading [####################]100.00% 77.2MB/77.2MB (10.1MB/s) 10s_video.zip.synapse_download_40879178 Done... Downloading [####################]100.00% 179.7kB/179.7kB (58.7MB/s) raw.png Done... Downloading [####################]100.00% 59.3MB/59.3MB (9.5MB/s) 10s_video.zip.synapse_download_40879147 Done... Downloading [####################]100.00% 250.0kB/250.0kB (106.9MB/s) raw.png Done... Downloading [####################]100.00% 93.8MB/93.8MB (4.4MB/s) 10s_video.zip.synapse_download_40879184 Done... Downloading [####################]100.00% 3.5kB/3.5kB (18.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 58.4MB/58.4MB (6.7MB/s) 10s_video.zip.synapse_download_42122415 Done... Downloading [####################]100.00% 322.8kB/322.8kB (69.1MB/s) raw.png Done... Downloading [####################]100.00% 65.2kB/65.2kB (130.2MB/s) 10s_video.zip Done... Downloading [####################]100.00% 1.6kB/1.6kB (10.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 84.3kB/84.3kB (68.4MB/s) raw.png Done... Downloading [####################]100.00% 97.9MB/97.9MB (2.7MB/s) 10s_video.zip.synapse_download_40879191 Done... Downloading [####################]100.00% 2.3kB/2.3kB (11.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 184.9kB/184.9kB (89.7MB/s) raw.png Done... Downloading [####################]100.00% 94.9MB/94.9MB (2.3MB/s) 10s_video.zip.synapse_download_42122435 Done... Downloading [####################]100.00% 333.9kB/333.9kB (64.5MB/s) raw.png Done... Downloading [####################]100.00% 1.6kB/1.6kB (10.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 312.4kB/312.4kB (3.5MB/s) raw.png Done... Downloading [####################]100.00% 67.2MB/67.2MB (7.0MB/s) 10s_video.zip.synapse_download_40879144 Done... Downloading [####################]100.00% 1.7kB/1.7kB (6.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 346.9kB/346.9kB (58.3MB/s) raw.png Done... Downloading [####################]100.00% 56.6MB/56.6MB (2.2MB/s) 10s_video.zip.synapse_download_42122428 Done... Downloading [####################]100.00% 346.8kB/346.8kB (998.4kB/s) raw.png Done... Downloading [####################]100.00% 68.4MB/68.4MB (3.5MB/s) 10s_video.zip.synapse_download_40879124 Done... Downloading [####################]100.00% 335.8kB/335.8kB (3.8MB/s) raw.png Done... Downloading [####################]100.00% 69.6MB/69.6MB (3.3MB/s) 10s_video.zip.synapse_download_42122406 Done... Downloading [####################]100.00% 1.2kB/1.2kB (4.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 240.1kB/240.1kB (55.3MB/s) raw.png Done... Downloading [####################]100.00% 76.0MB/76.0MB (2.6MB/s) 10s_video.zip.synapse_download_40879130 Done... Downloading [####################]100.00% 248.7kB/248.7kB (681.3kB/s) raw.png Done... Downloading [####################]100.00% 81.1MB/81.1MB (4.1MB/s) 10s_video.zip.synapse_download_42122398 Done... Downloading [####################]100.00% 348.9kB/348.9kB (3.7MB/s) raw.png Done... Downloading [####################]100.00% 82.7MB/82.7MB (2.7MB/s) 10s_video.zip.synapse_download_42122413 Done... Downloading [####################]100.00% 3.2kB/3.2kB (7.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 255.0kB/255.0kB (53.9MB/s) raw.png Done... Downloading [####################]100.00% 82.2MB/82.2MB (2.2MB/s) 10s_video.zip.synapse_download_40879119 Done... Downloading [####################]100.00% 1.3kB/1.3kB (2.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 321.3kB/321.3kB (75.2MB/s) raw.png Done... Downloading [####################]100.00% 91.1MB/91.1MB (2.9MB/s) 10s_video.zip.synapse_download_40879150 Done... Downloading [####################]100.00% 2.0kB/2.0kB (5.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 309.6kB/309.6kB (889.2kB/s) raw.png Done... Downloading [####################]100.00% 88.6MB/88.6MB (4.2MB/s) 10s_video.zip.synapse_download_42122397 Done... Downloading [####################]100.00% 1.3kB/1.3kB (6.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 293.9kB/293.9kB (3.3MB/s) raw.png Done... Downloading [####################]100.00% 79.0MB/79.0MB (4.6MB/s) 10s_video.zip.synapse_download_40879117 Done... Downloading [####################]100.00% 3.5kB/3.5kB (20.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 318.9kB/318.9kB (68.4MB/s) raw.png Done... Downloading [####################]100.00% 79.2MB/79.2MB (2.2MB/s) 10s_video.zip.synapse_download_42122393 Done... Downloading [####################]100.00% 1.2kB/1.2kB (2.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 238.5kB/238.5kB (83.0MB/s) raw.png Done... Downloading [####################]100.00% 82.2MB/82.2MB (7.1MB/s) 10s_video.zip.synapse_download_40879112 Done... Downloading [####################]100.00% 2.8kB/2.8kB (5.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 341.7kB/341.7kB (989.6kB/s) raw.png Done... Downloading [####################]100.00% 81.8MB/81.8MB (4.2MB/s) 10s_video.zip.synapse_download_40879129 Done... Downloading [####################]100.00% 2.7kB/2.7kB (7.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 436.6kB/436.6kB (4.8MB/s) raw.png Done... Downloading [####################]100.00% 97.0MB/97.0MB (5.1MB/s) 10s_video.zip.synapse_download_40879170 Done... Downloading [####################]100.00% 377.5kB/377.5kB (59.8MB/s) raw.png Done... Downloading [####################]100.00% 69.5MB/69.5MB (1.6MB/s) 10s_video.zip.synapse_download_40879128 Done... Downloading [####################]100.00% 2.8kB/2.8kB (9.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 322.9kB/322.9kB (64.0MB/s) raw.png Done... Downloading [####################]100.00% 85.1MB/85.1MB (1.9MB/s) 10s_video.zip.synapse_download_40879116 Done... Downloading [####################]100.00% 2.3kB/2.3kB (6.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 345.7kB/345.7kB (57.0MB/s) raw.png Done... Downloading [####################]100.00% 89.2MB/89.2MB (5.5MB/s) 10s_video.zip.synapse_download_42122382 Done... Downloading [####################]100.00% 1.9kB/1.9kB (6.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 384.5kB/384.5kB (555.0kB/s) raw.png Done... Downloading [####################]100.00% 97.2MB/97.2MB (2.8MB/s) 10s_video.zip.synapse_download_40879098 Done... Downloading [####################]100.00% 2.6kB/2.6kB (9.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 80.8MB/80.8MB (4.5MB/s) 10s_video.zip.synapse_download_40879064 Done... Downloading [####################]100.00% 405.2kB/405.2kB (935.4kB/s) raw.png Done... Downloading [####################]100.00% 1.5kB/1.5kB (2.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 362.0kB/362.0kB (1.0MB/s) raw.png Done... Downloading [####################]100.00% 100.9MB/100.9MB (3.6MB/s) 10s_video.zip.synapse_download_40879083 Done... Downloading [####################]100.00% 398.1kB/398.1kB (917.4kB/s) raw.png Done... Downloading [####################]100.00% 87.7MB/87.7MB (3.6MB/s) 10s_video.zip.synapse_download_40879056 Done... Downloading [####################]100.00% 2.2kB/2.2kB (2.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 361.6kB/361.6kB (1.0MB/s) raw.png Done... Downloading [####################]100.00% 81.1MB/81.1MB (5.3MB/s) 10s_video.zip.synapse_download_40879044 Done... Downloading [####################]100.00% 83.0MB/83.0MB (3.2MB/s) 10s_video.zip.synapse_download_40879051 Done... Downloading [####################]100.00% 93.1MB/93.1MB (8.2MB/s) 10s_video.zip.synapse_download_42122379 Done... Downloading [####################]100.00% 1.9kB/1.9kB (2.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 457.0kB/457.0kB (749.1kB/s) raw.png Done... Downloading [####################]100.00% 1.5kB/1.5kB (5.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 367.2kB/367.2kB (1.0MB/s) raw.png Done... Downloading [####################]100.00% 1.7kB/1.7kB (3.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 438.6kB/438.6kB (4.8MB/s) raw.png Done... Downloading [####################]100.00% 86.5MB/86.5MB (3.7MB/s) 10s_video.zip.synapse_download_40879048 Done... Downloading [####################]100.00% 2.0kB/2.0kB (5.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 410.4kB/410.4kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 102.7MB/102.7MB (4.6MB/s) 10s_video.zip.synapse_download_40879049 Done... Downloading [####################]100.00% 88.0MB/88.0MB (2.6MB/s) 10s_video.zip.synapse_download_42122384 Done... Downloading [####################]100.00% 1.7kB/1.7kB (8.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 256.1kB/256.1kB (73.8MB/s) raw.png Done... Downloading [####################]100.00% 2.1kB/2.1kB (7.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 413.5kB/413.5kB (4.3MB/s) raw.png Done... Downloading [####################]100.00% 65.2kB/65.2kB (122.2MB/s) 10s_video.zip Done... Downloading [####################]100.00% 87.8MB/87.8MB (4.4MB/s) 10s_video.zip.synapse_download_40879042 Done... Downloading [####################]100.00% 86.6MB/86.6MB (3.9MB/s) 10s_video.zip.synapse_download_40879041 Done... Downloading [####################]100.00% 3.3kB/3.3kB (15.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 85.1kB/85.1kB (70.4MB/s) raw.png Done... Downloading [####################]100.00% 1.7kB/1.7kB (4.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 411.8kB/411.8kB (10.5MB/s) raw.png Done... Downloading [####################]100.00% 3.1kB/3.1kB (9.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 369.2kB/369.2kB (3.0MB/s) raw.png Done... Downloading [####################]100.00% 92.2MB/92.2MB (6.5MB/s) 10s_video.zip.synapse_download_40879017 Done... Downloading [####################]100.00% 2.3kB/2.3kB (12.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 80.7MB/80.7MB (4.5MB/s) 10s_video.zip.synapse_download_40879016 Done... Downloading [####################]100.00% 261.0kB/261.0kB (474.4kB/s) raw.png Done... Downloading [####################]100.00% 1.2kB/1.2kB (7.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 64.7MB/64.7MB (5.5MB/s) 10s_video.zip.synapse_download_40878983 Done... Downloading [####################]100.00% 254.2kB/254.2kB (684.9kB/s) raw.png Done... Downloading [####################]100.00% 1.8kB/1.8kB (10.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 333.7kB/333.7kB (900.5kB/s) raw.png Done... Downloading [####################]100.00% 80.2MB/80.2MB (4.1MB/s) 10s_video.zip.synapse_download_40878987 Done... Downloading [####################]100.00% 240.0kB/240.0kB (516.9kB/s) raw.png Done... Downloading [####################]100.00% 64.7MB/64.7MB (7.0MB/s) 10s_video.zip.synapse_download_42122372 Done... Downloading [####################]100.00% 1.7kB/1.7kB (11.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 240.8kB/240.8kB (874.8kB/s) raw.png Done... Downloading [####################]100.00% 71.0MB/71.0MB (4.8MB/s) 10s_video.zip.synapse_download_40878976 Done... Downloading [####################]100.00% 97.5MB/97.5MB (1.9MB/s) 10s_video.zip.synapse_download_40879034 Done... Downloading [####################]100.00% 2.0kB/2.0kB (12.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 351.6kB/351.6kB (965.5kB/s) raw.png Done... Downloading [####################]100.00% 1.1kB/1.1kB (7.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 319.7kB/319.7kB (3.4MB/s) raw.png Done... Downloading [####################]100.00% 98.3MB/98.3MB (2.9MB/s) 10s_video.zip.synapse_download_40879036 Done... Downloading [####################]100.00% 64.5MB/64.5MB (3.1MB/s) 10s_video.zip.synapse_download_40879001 Done... Downloading [####################]100.00% 3.0kB/3.0kB (20.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 269.0kB/269.0kB (99.7MB/s) raw.png Done... Downloading [####################]100.00% 1.1kB/1.1kB (6.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 329.9kB/329.9kB (1.4MB/s) raw.png Done... Downloading [####################]100.00% 60.6MB/60.6MB (3.5MB/s) 10s_video.zip.synapse_download_40878974 Done... Downloading [####################]100.00% 3.0kB/3.0kB (12.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 297.0kB/297.0kB (107.4MB/s) raw.png Done... Downloading [####################]100.00% 78.0MB/78.0MB (4.1MB/s) 10s_video.zip.synapse_download_40878966 Done... Downloading [####################]100.00% 71.8MB/71.8MB (6.4MB/s) 10s_video.zip.synapse_download_42122365 Done... Downloading [####################]100.00% 74.8MB/74.8MB (3.0MB/s) 10s_video.zip.synapse_download_40878968 Done... Downloading [####################]100.00% 220.7kB/220.7kB (786.1kB/s) raw.png Done... Downloading [####################]100.00% 4.0kB/4.0kB (25.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 257.1kB/257.1kB (516.2kB/s) raw.png Done... Downloading [####################]100.00% 1.5kB/1.5kB (6.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 288.7kB/288.7kB (790.7kB/s) raw.png Done... Downloading [####################]100.00% 86.3MB/86.3MB (3.7MB/s) 10s_video.zip.synapse_download_40878947 Done... Downloading [####################]100.00% 982.0bytes/982.0bytes (6.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 275.3kB/275.3kB (995.7kB/s) raw.png Done... Downloading [####################]100.00% 69.0MB/69.0MB (2.5MB/s) 10s_video.zip.synapse_download_40878978 Done... Downloading [####################]100.00% 1.0kB/1.0kB (6.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 260.6kB/260.6kB (949.9kB/s) raw.png Done... Downloading [####################]100.00% 66.9MB/66.9MB (4.3MB/s) 10s_video.zip.synapse_download_40878945 Done... Downloading [####################]100.00% 2.5kB/2.5kB (11.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 303.3kB/303.3kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 78.1MB/78.1MB (2.0MB/s) 10s_video.zip.synapse_download_40878975 Done... Downloading [####################]100.00% 3.8kB/3.8kB (24.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 264.1kB/264.1kB (964.5kB/s) raw.png Done... Downloading [####################]100.00% 64.3MB/64.3MB (3.7MB/s) 10s_video.zip.synapse_download_40878930 Done... Downloading [####################]100.00% 2.2kB/2.2kB (11.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 73.0MB/73.0MB (4.5MB/s) 10s_video.zip.synapse_download_40878952 Done... Downloading [####################]100.00% 394.8kB/394.8kB (1.0MB/s) raw.png Done... Downloading [####################]100.00% 4.3kB/4.3kB (16.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 232.5kB/232.5kB (1.2MB/s) raw.png Done... Downloading [####################]100.00% 65.2MB/65.2MB (3.7MB/s) 10s_video.zip.synapse_download_40878922 Done... Downloading [####################]100.00% 40.5MB/40.5MB (8.9MB/s) 10s_video.zip.synapse_download_40878900 Done... Downloading [####################]100.00% 1.7kB/1.7kB (12.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 223.9kB/223.9kB (1.2MB/s) raw.png Done... Downloading [####################]100.00% 2.9kB/2.9kB (19.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 72.2MB/72.2MB (6.8MB/s) 10s_video.zip.synapse_download_40878913 Done... Downloading [####################]100.00% 196.4kB/196.4kB (1.0MB/s) raw.png Done... Downloading [####################]100.00% 3.5kB/3.5kB (15.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 367.8kB/367.8kB (1002.3kB/s) raw.png Done... Downloading [####################]100.00% 65.2kB/65.2kB (82.2MB/s) 10s_video.zip Done... Downloading [####################]100.00% 2.9kB/2.9kB (17.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 88.7kB/88.7kB (979.9kB/s) raw.png Done... Downloading [####################]100.00% 68.7MB/68.7MB (12.0MB/s) 10s_video.zip.synapse_download_40878901 Done... Downloading [####################]100.00% 1.5kB/1.5kB (7.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 344.1kB/344.1kB (957.5kB/s) raw.png Done... Downloading [####################]100.00% 82.0MB/82.0MB (6.6MB/s) 10s_video.zip.synapse_download_40878888 Done... Downloading [####################]100.00% 59.1MB/59.1MB (2.8MB/s) 10s_video.zip.synapse_download_40878904 Done... Downloading [####################]100.00% 73.6MB/73.6MB (2.4MB/s) 10s_video.zip.synapse_download_40878915 Done... Downloading [####################]100.00% 3.8kB/3.8kB (7.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 2.4kB/2.4kB (4.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 405.8kB/405.8kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 220.3kB/220.3kB (76.7MB/s) raw.png Done... Downloading [####################]100.00% 3.0kB/3.0kB (18.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 356.3kB/356.3kB (3.8MB/s) raw.png Done... Downloading [####################]100.00% 89.6MB/89.6MB (3.1MB/s) 10s_video.zip.synapse_download_40878914 Done... Downloading [####################]100.00% 3.1kB/3.1kB (17.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 80.8MB/80.8MB (6.3MB/s) 10s_video.zip.synapse_download_40878875 Done... Downloading [####################]100.00% 393.1kB/393.1kB (92.2MB/s) raw.png Done... Downloading [####################]100.00% 65.2kB/65.2kB (53.7MB/s) 10s_video.zip Done... Downloading [####################]100.00% 295.6kB/295.6kB (799.8kB/s) raw.png Done... Downloading [####################]100.00% 1.7kB/1.7kB (1.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 450.1kB/450.1kB (4.7MB/s) raw.png Done... Downloading [####################]100.00% 54.9MB/54.9MB (6.9MB/s) 10s_video.zip.synapse_download_42122351 Done... Downloading [####################]100.00% 79.3MB/79.3MB (4.1MB/s) 10s_video.zip.synapse_download_40878872 Done... Downloading [####################]100.00% 1.9kB/1.9kB (9.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 417.6kB/417.6kB (4.2MB/s) raw.png Done... Downloading [####################]100.00% 1.0kB/1.0kB (3.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 270.0kB/270.0kB (73.5MB/s) raw.png Done... Downloading [####################]100.00% 104.9MB/104.9MB (7.1MB/s) 10s_video.zip.synapse_download_40878899 Done... Downloading [####################]100.00% 4.0kB/4.0kB (8.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 412.2kB/412.2kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 106.5MB/106.5MB (7.2MB/s) 10s_video.zip.synapse_download_40878853 Done... Downloading [####################]100.00% 3.4kB/3.4kB (13.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 78.7MB/78.7MB (3.2MB/s) 10s_video.zip.synapse_download_40878869 Done... Downloading [####################]100.00% 423.5kB/423.5kB (4.8MB/s) raw.png Done... Downloading [####################]100.00% 1.8kB/1.8kB (4.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 378.5kB/378.5kB (78.8MB/s) raw.png Done... Downloading [####################]100.00% 73.9MB/73.9MB (4.1MB/s) 10s_video.zip.synapse_download_40878849 Done... Downloading [####################]100.00% 2.0kB/2.0kB (4.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 393.7kB/393.7kB (79.1MB/s) raw.png Done... Downloading [####################]100.00% 84.7MB/84.7MB (3.1MB/s) 10s_video.zip.synapse_download_40878855 Done... Downloading [####################]100.00% 713.0bytes/713.0bytes (159.6kB/s) instrument_instances.png Done... Downloading [####################]100.00% 443.2kB/443.2kB (57.4MB/s) raw.png Done... Downloading [####################]100.00% 95.8MB/95.8MB (4.2MB/s) 10s_video.zip.synapse_download_40878858 Done... Downloading [####################]100.00% 2.6kB/2.6kB (14.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 449.7kB/449.7kB (73.0MB/s) raw.png Done... Downloading [####################]100.00% 97.1MB/97.1MB (4.2MB/s) 10s_video.zip.synapse_download_40878856 Done... Downloading [####################]100.00% 86.2MB/86.2MB (3.9MB/s) 10s_video.zip.synapse_download_40878844 Done... Downloading [####################]100.00% 3.0kB/3.0kB (12.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 468.1kB/468.1kB (542.1kB/s) raw.png Done... Downloading [####################]100.00% 1.8kB/1.8kB (5.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 337.7kB/337.7kB (771.8kB/s) raw.png Done... Downloading [####################]100.00% 88.1MB/88.1MB (2.6MB/s) 10s_video.zip.synapse_download_40878866 Done... Downloading [####################]100.00% 85.9MB/85.9MB (6.8MB/s) 10s_video.zip.synapse_download_40878792 Done... Downloading [####################]100.00% 1.4kB/1.4kB (4.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 384.0kB/384.0kB (597.1kB/s) raw.png Done... Downloading [####################]100.00% 2.0kB/2.0kB (11.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 103.1MB/103.1MB (2.9MB/s) 10s_video.zip.synapse_download_42122342 Done... Downloading [####################]100.00% 378.7kB/378.7kB (826.4kB/s) raw.png Done... Downloading [####################]100.00% 2.6kB/2.6kB (7.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 351.8kB/351.8kB (940.3kB/s) raw.png Done... Downloading [####################]100.00% 100.2MB/100.2MB (3.0MB/s) 10s_video.zip.synapse_download_40878820 Done... Downloading [####################]100.00% 1.8kB/1.8kB (2.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 406.8kB/406.8kB (891.3kB/s) raw.png Done... Downloading [####################]100.00% 89.0MB/89.0MB (1.9MB/s) 10s_video.zip.synapse_download_40878818 Done... Downloading [####################]100.00% 1.3kB/1.3kB (5.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 395.9kB/395.9kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 89.7MB/89.7MB (4.1MB/s) 10s_video.zip.synapse_download_40878789 Done... Downloading [####################]100.00% 1.7kB/1.7kB (2.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 269.5kB/269.5kB (59.7MB/s) raw.png Done... Downloading [####################]100.00% 69.0MB/69.0MB (7.7MB/s) 10s_video.zip.synapse_download_40878775 Done... Downloading [####################]100.00% 101.3MB/101.3MB (2.7MB/s) 10s_video.zip.synapse_download_40878834 Done... Downloading [####################]100.00% 1.1kB/1.1kB (1.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 398.5kB/398.5kB (4.4MB/s) raw.png Done... Downloading [####################]100.00% 2.0kB/2.0kB (8.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 407.1kB/407.1kB (35.7MB/s) raw.png Done... Downloading [####################]100.00% 98.8MB/98.8MB (4.0MB/s) 10s_video.zip.synapse_download_40878787 Done... Downloading [####################]100.00% 1.5kB/1.5kB (5.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 398.5kB/398.5kB (50.9MB/s) raw.png Done... Downloading [####################]100.00% 113.0MB/113.0MB (4.4MB/s) 10s_video.zip.synapse_download_40878778 Done... Downloading [####################]100.00% 1.8kB/1.8kB (4.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 362.7kB/362.7kB (4.0MB/s) raw.png Done... Downloading [####################]100.00% 105.3MB/105.3MB (2.9MB/s) 10s_video.zip.synapse_download_40878788 Done... Downloading [####################]100.00% 1.6kB/1.6kB (6.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 396.5kB/396.5kB (4.4MB/s) raw.png Done... Downloading [####################]100.00% 99.5MB/99.5MB (8.6MB/s) 10s_video.zip.synapse_download_40878784 Done... Downloading [####################]100.00% 1.8kB/1.8kB (4.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 459.0kB/459.0kB (2.5MB/s) raw.png Done... Downloading [####################]100.00% 105.4MB/105.4MB (4.7MB/s) 10s_video.zip.synapse_download_40878803 Done... \ Downloaded 10.0GB (19.7MB/s)2021-01-08 10:35:58,179 [retry:52 - DEBUG]: calling
.
at 0x7ff765bf43a0> resulted in an Exception 2021-01-08 10:35:58,179 [retry:86 - DEBUG]: retrying exception: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) 2021-01-08 10:35:58,179 [retry:92 - DEBUG]: total wait time 0 seconds ... Retrying in 0.6 seconds... Downloading [####################]100.00% 3.1kB/3.1kB (11.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 93.2MB/93.2MB (6.4MB/s) 10s_video.zip.synapse_download_40878755 Done... Downloading [####################]100.00% 376.5kB/376.5kB (1.0MB/s) raw.png Done... Downloading [####################]100.00% 1.8kB/1.8kB (3.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 189.1kB/189.1kB (42.8MB/s) raw.png Done... Downloading [####################]100.00% 106.2MB/106.2MB (2.6MB/s) 10s_video.zip.synapse_download_42122334 Done... Downloading [####################]100.00% 1.5kB/1.5kB (782.5kB/s) instrument_instances.png Done... Downloading [####################]100.00% 463.4kB/463.4kB (461.8kB/s) raw.png Done... Downloading [####################]100.00% 114.5MB/114.5MB (2.9MB/s) 10s_video.zip.synapse_download_40878780 Done... Downloading [####################]100.00% 1.2kB/1.2kB (7.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 100.1MB/100.1MB (6.1MB/s) 10s_video.zip.synapse_download_40878717 Done... Downloading [####################]100.00% 321.1kB/321.1kB (589.2kB/s) raw.png Done... Downloading [####################]100.00% 98.9MB/98.9MB (3.4MB/s) 10s_video.zip.synapse_download_40878749 Done... Downloading [####################]100.00% 1.3kB/1.3kB (3.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 236.2kB/236.2kB (650.8kB/s) raw.png Done... Downloading [####################]100.00% 96.3MB/96.3MB (3.6MB/s) 10s_video.zip.synapse_download_40878725 Done... Downloading [####################]100.00% 2.3kB/2.3kB (5.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 315.9kB/315.9kB (694.0kB/s) raw.png Done... Downloading [####################]100.00% 3.4kB/3.4kB (4.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 461.3kB/461.3kB (716.3kB/s) raw.png Done... Downloading [####################]100.00% 74.1MB/74.1MB (4.1MB/s) 10s_video.zip.synapse_download_40878726 Done... Downloading [####################]100.00% 2.1kB/2.1kB (6.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 345.6kB/345.6kB (199.5kB/s) raw.png Done... Downloading [####################]100.00% 88.6MB/88.6MB (3.9MB/s) 10s_video.zip.synapse_download_40878711 Done... Downloading [####################]100.00% 2.6kB/2.6kB (6.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 515.6kB/515.6kB (741.1kB/s) raw.png Done... Downloading [####################]100.00% 88.0MB/88.0MB (4.3MB/s) 10s_video.zip.synapse_download_40878709 Done... Downloading [####################]100.00% 2.2kB/2.2kB (4.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 431.5kB/431.5kB (1.2MB/s) raw.png Done... Downloading [####################]100.00% 95.7MB/95.7MB (2.6MB/s) 10s_video.zip.synapse_download_40878715 Done... Downloading [####################]100.00% 2.1kB/2.1kB (6.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 489.4kB/489.4kB (1.0MB/s) raw.png Done... Downloading [####################]100.00% 102.9MB/102.9MB (8.3MB/s) 10s_video.zip.synapse_download_40878685 Done... Downloading [####################]100.00% 2.1kB/2.1kB (7.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 530.9kB/530.9kB (5.6MB/s) raw.png Done... Downloading [####################]100.00% 117.7MB/117.7MB (2.0MB/s) 10s_video.zip.synapse_download_40878733 Done... Downloading [####################]100.00% 2.2kB/2.2kB (5.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 510.3kB/510.3kB (77.4MB/s) raw.png Done... Downloading [####################]100.00% 108.0MB/108.0MB (2.5MB/s) 10s_video.zip.synapse_download_40878705 Done... Downloading [####################]100.00% 1.4kB/1.4kB (9.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 387.7kB/387.7kB (2.1MB/s) raw.png Done... Downloading [####################]100.00% 117.9MB/117.9MB (3.0MB/s) 10s_video.zip.synapse_download_40878710 Done... Downloading [####################]100.00% 2.0kB/2.0kB (4.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 455.6kB/455.6kB (624.2kB/s) raw.png Done... Downloading [####################]100.00% 112.8MB/112.8MB (2.5MB/s) 10s_video.zip.synapse_download_40878672 Done... Downloading [####################]100.00% 3.1kB/3.1kB (7.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 415.9kB/415.9kB (1.2MB/s) raw.png Done... Downloading [####################]100.00% 125.9MB/125.9MB (3.3MB/s) 10s_video.zip.synapse_download_40878669 Done... Downloading [####################]100.00% 2.5kB/2.5kB (14.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 430.1kB/430.1kB (4.7MB/s) raw.png Done... Downloading [####################]100.00% 120.1MB/120.1MB (3.2MB/s) 10s_video.zip.synapse_download_40878666 Done... Downloading [####################]100.00% 2.0kB/2.0kB (11.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 398.9kB/398.9kB (63.2MB/s) raw.png Done... Downloading [####################]100.00% 103.5MB/103.5MB (2.8MB/s) 10s_video.zip.synapse_download_40878657 Done... Downloading [####################]100.00% 2.5kB/2.5kB (5.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 428.7kB/428.7kB (68.8MB/s) raw.png Done... Downloading [####################]100.00% 105.4MB/105.4MB (4.4MB/s) 10s_video.zip.synapse_download_40878676 Done... Downloading [####################]100.00% 3.4kB/3.4kB (3.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 533.7kB/533.7kB (5.6MB/s) raw.png Done... Downloading [####################]100.00% 107.1MB/107.1MB (2.2MB/s) 10s_video.zip.synapse_download_40878665 Done... Downloading [####################]100.00% 2.1kB/2.1kB (4.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 459.1kB/459.1kB (1.3MB/s) raw.png Done... Downloading [####################]100.00% 95.7MB/95.7MB (3.0MB/s) 10s_video.zip.synapse_download_40878682 Done... Downloading [####################]100.00% 1.6kB/1.6kB (4.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 294.4kB/294.4kB (802.1kB/s) raw.png Done... Downloading [####################]100.00% 119.2MB/119.2MB (4.1MB/s) 10s_video.zip.synapse_download_40878651 Done... Downloading [####################]100.00% 2.6kB/2.6kB (8.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 301.8kB/301.8kB (866.4kB/s) raw.png Done... Downloading [####################]100.00% 108.0MB/108.0MB (4.6MB/s) 10s_video.zip.synapse_download_40878624 Done... Downloading [####################]100.00% 97.5MB/97.5MB (2.3MB/s) 10s_video.zip.synapse_download_40878659 Done... Downloading [####################]100.00% 2.5kB/2.5kB (16.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 127.8MB/127.8MB (3.1MB/s) 10s_video.zip.synapse_download_40878652 Done... Downloading [####################]100.00% 510.7kB/510.7kB (5.6MB/s) raw.png Done... Downloading [####################]100.00% 3.0kB/3.0kB (7.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 454.3kB/454.3kB (76.8MB/s) raw.png Done... Downloading [####################]100.00% 460.8kB/460.8kB (52.6MB/s) raw.png Done... Downloading [####################]100.00% 73.5MB/73.5MB (2.7MB/s) 10s_video.zip.synapse_download_40878439 Done... Downloading [####################]100.00% 2.0kB/2.0kB (10.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 394.3kB/394.3kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 124.1MB/124.1MB (1.8MB/s) 10s_video.zip.synapse_download_40878650 Done... Downloading [####################]100.00% 2.1kB/2.1kB (11.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 501.7kB/501.7kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 105.1MB/105.1MB (3.3MB/s) 10s_video.zip.synapse_download_42122333 Done... Downloading [####################]100.00% 124.4MB/124.4MB (3.2MB/s) 10s_video.zip.synapse_download_40878434 Done... Downloading [####################]100.00% 2.9kB/2.9kB (14.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 431.0kB/431.0kB (4.8MB/s) raw.png Done... Downloading [####################]100.00% 1.1kB/1.1kB (4.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 117.4MB/117.4MB (3.3MB/s) 10s_video.zip.synapse_download_40878621 Done... Downloading [####################]100.00% 549.7kB/549.7kB (6.0MB/s) raw.png Done... Downloading [####################]100.00% 2.1kB/2.1kB (7.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 534.6kB/534.6kB (5.7MB/s) raw.png Done... Downloading [####################]100.00% 104.0MB/104.0MB (2.2MB/s) 10s_video.zip.synapse_download_40878420 Done... Downloading [####################]100.00% 2.2kB/2.2kB (7.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 506.6kB/506.6kB (1.4MB/s) raw.png Done... Downloading [####################]100.00% 111.9MB/111.9MB (2.9MB/s) 10s_video.zip.synapse_download_40878412 Done... Downloading [####################]100.00% 3.6kB/3.6kB (18.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 388.8kB/388.8kB (95.0MB/s) raw.png Done... Downloading [####################]100.00% 116.9MB/116.9MB (6.0MB/s) 10s_video.zip.synapse_download_40878391 Done... Downloading [####################]100.00% 2.0kB/2.0kB (12.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 542.2kB/542.2kB (5.9MB/s) raw.png Done... Downloading [####################]100.00% 112.3MB/112.3MB (4.3MB/s) 10s_video.zip.synapse_download_40878407 Done... Downloading [####################]100.00% 95.2MB/95.2MB (3.1MB/s) 10s_video.zip.synapse_download_40878424 Done... Downloading [####################]100.00% 1.1kB/1.1kB (7.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 524.7kB/524.7kB (80.9MB/s) raw.png Done... Downloading [####################]100.00% 3.1kB/3.1kB (13.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 447.0kB/447.0kB (99.0MB/s) raw.png Done... Downloading [####################]100.00% 117.1MB/117.1MB (3.5MB/s) 10s_video.zip.synapse_download_40878409 Done... Downloading [####################]100.00% 2.0kB/2.0kB (10.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 403.7kB/403.7kB (586.1kB/s) raw.png Done... Downloading [####################]100.00% 80.5MB/80.5MB (2.3MB/s) 10s_video.zip.synapse_download_40878372 Done... Downloading [####################]100.00% 1.7kB/1.7kB (9.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 450.3kB/450.3kB (1.2MB/s) raw.png Done... Downloading [####################]100.00% 123.9MB/123.9MB (3.6MB/s) 10s_video.zip.synapse_download_40878404 Done... Downloading [####################]100.00% 1.6kB/1.6kB (9.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 527.0kB/527.0kB (1016.6kB/s) raw.png Done... Downloading [####################]100.00% 98.8MB/98.8MB (2.7MB/s) 10s_video.zip.synapse_download_40878386 Done... Downloading [####################]100.00% 2.1kB/2.1kB (12.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 439.9kB/439.9kB (4.9MB/s) raw.png Done... Downloading [####################]100.00% 133.5MB/133.5MB (2.9MB/s) 10s_video.zip.synapse_download_40878387 Done... Downloading [####################]100.00% 2.2kB/2.2kB (12.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 354.6kB/354.6kB (3.8MB/s) raw.png Done... Downloading [####################]100.00% 115.8MB/115.8MB (3.2MB/s) 10s_video.zip.synapse_download_40878379 Done... Downloading [####################]100.00% 2.9kB/2.9kB (15.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 451.1kB/451.1kB (2.5MB/s) raw.png Done... Downloading [####################]100.00% 116.2MB/116.2MB (2.2MB/s) 10s_video.zip.synapse_download_40878390 Done... Downloading [####################]100.00% 267.3kB/267.3kB (2.7MB/s) raw.png Done... Downloading [####################]100.00% 64.1MB/64.1MB (10.8MB/s) 10s_video.zip.synapse_download_42122319 Done... Downloading [####################]100.00% 261.8kB/261.8kB (2.9MB/s) raw.png Done... Downloading [####################]100.00% 106.3MB/106.3MB (3.9MB/s) 10s_video.zip.synapse_download_40878346 Done... Downloading [####################]100.00% 1.9kB/1.9kB (11.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 115.6kB/115.6kB (605.0kB/s) raw.png Done... Downloading [####################]100.00% 65.2kB/65.2kB (709.2kB/s) 10s_video.zip Done... Downloading [####################]100.00% 124.3MB/124.3MB (2.9MB/s) 10s_video.zip.synapse_download_40878348 Done... Downloading [####################]100.00% 2.3kB/2.3kB (13.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 92.6kB/92.6kB (1000.7kB/s) raw.png Done... Downloading [####################]100.00% 92.8MB/92.8MB (5.3MB/s) 10s_video.zip.synapse_download_42122321 Done... Downloading [####################]100.00% 2.8kB/2.8kB (18.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 393.6kB/393.6kB (852.1kB/s) raw.png Done... Downloading [####################]100.00% 113.7MB/113.7MB (4.1MB/s) 10s_video.zip.synapse_download_40878352 Done... Downloading [####################]100.00% 380.3kB/380.3kB (1007.8kB/s) raw.png Done... Downloading [####################]100.00% 1.7kB/1.7kB (6.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 6.0MB/6.0MB (6.4MB/s) 10s_video.zip Done... Downloading [####################]100.00% 94.4MB/94.4MB (5.5MB/s) 10s_video.zip.synapse_download_40878335 Done... Downloading [####################]100.00% 346.2kB/346.2kB (97.5MB/s) raw.png Done... Downloading [####################]100.00% 3.3kB/3.3kB (5.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 280.3kB/280.3kB (1011.2kB/s) raw.png Done... Downloading [####################]100.00% 108.1MB/108.1MB (2.7MB/s) 10s_video.zip.synapse_download_40878344 Done... Downloading [####################]100.00% 2.7kB/2.7kB (17.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 299.8kB/299.8kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 762.0bytes/762.0bytes (866.5kB/s) instrument_instances.png Done... Downloading [####################]100.00% 315.5kB/315.5kB (1.1MB/s) raw.png Done... Downloading [####################]100.00% 80.0MB/80.0MB (8.2MB/s) 10s_video.zip.synapse_download_40878321 Done... / Downloaded 15.1GB (19.0MB/s)2021-01-08 10:40:50,027 [retry:52 - DEBUG]: calling
.
at 0x7ff764133310> resulted in an Exception 2021-01-08 10:40:50,028 [retry:86 - DEBUG]: retrying exception: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) 2021-01-08 10:40:50,028 [retry:92 - DEBUG]: total wait time 0 seconds ... Retrying in 1.3 seconds... Downloading [####################]100.00% 87.3MB/87.3MB (6.8MB/s) 10s_video.zip.synapse_download_40878322 Done... Downloading [####################]100.00% 1.3kB/1.3kB (3.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 281.4kB/281.4kB (772.2kB/s) raw.png Done... Downloading [####################]100.00% 303.4kB/303.4kB (3.2MB/s) raw.png Done... Downloading [####################]100.00% 58.1MB/58.1MB (8.9MB/s) 10s_video.zip.synapse_download_42122314 Done... Downloading [####################]100.00% 1.5kB/1.5kB (3.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 292.3kB/292.3kB (799.1kB/s) raw.png Done... Downloading [####################]100.00% 94.4MB/94.4MB (3.8MB/s) 10s_video.zip.synapse_download_40878302 Done... Downloading [####################]100.00% 277.8kB/277.8kB (789.8kB/s) raw.png Done... Downloading [####################]100.00% 96.7MB/96.7MB (3.2MB/s) 10s_video.zip.synapse_download_40878336 Done... Downloading [####################]100.00% 2.8kB/2.8kB (4.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 298.9kB/298.9kB (3.3MB/s) raw.png Done... Downloading [####################]100.00% 90.2MB/90.2MB (3.0MB/s) 10s_video.zip.synapse_download_40878313 Done... Downloading [####################]100.00% 3.4kB/3.4kB (9.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 393.6kB/393.6kB (4.3MB/s) raw.png Done... Downloading [####################]100.00% 95.4MB/95.4MB (9.4MB/s) 10s_video.zip.synapse_download_40878290 Done... Downloading [####################]100.00% 2.8kB/2.8kB (4.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 197.4kB/197.4kB (83.5MB/s) raw.png Done... Downloading [####################]100.00% 87.7MB/87.7MB (2.7MB/s) 10s_video.zip.synapse_download_40878316 Done... Downloading [####################]100.00% 2.1kB/2.1kB (11.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 287.9kB/287.9kB (3.1MB/s) raw.png Done... Downloading [####################]100.00% 83.9MB/83.9MB (6.7MB/s) 10s_video.zip.synapse_download_40878286 Done... Downloading [####################]100.00% 94.0MB/94.0MB (3.4MB/s) 10s_video.zip.synapse_download_42122302 Done... Downloading [####################]100.00% 3.2kB/3.2kB (17.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 201.1kB/201.1kB (2.2MB/s) raw.png Done... Downloading [####################]100.00% 1.8kB/1.8kB (5.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 309.2kB/309.2kB (3.1MB/s) raw.png Done... Downloading [####################]100.00% 86.8MB/86.8MB (1.7MB/s) 10s_video.zip.synapse_download_40878310 Done... Downloading [####################]100.00% 1.9kB/1.9kB (5.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 348.8kB/348.8kB (3.7MB/s) raw.png Done... Downloading [####################]100.00% 96.7MB/96.7MB (3.9MB/s) 10s_video.zip.synapse_download_40878268 Done... Downloading [####################]100.00% 1.1kB/1.1kB (3.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 175.5kB/175.5kB (84.7MB/s) raw.png Done... Downloading [####################]100.00% 90.7MB/90.7MB (8.7MB/s) 10s_video.zip.synapse_download_40878259 Done... Downloading [####################]100.00% 2.0kB/2.0kB (6.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 414.9kB/414.9kB (571.6kB/s) raw.png Done... Downloading [####################]100.00% 100.0MB/100.0MB (3.2MB/s) 10s_video.zip.synapse_download_40878276 Done... Downloading [####################]100.00% 1.9kB/1.9kB (7.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 259.2kB/259.2kB (568.1kB/s) raw.png Done... Downloading [####################]100.00% 90.7MB/90.7MB (3.7MB/s) 10s_video.zip.synapse_download_40878262 Done... Downloading [####################]100.00% 2.5kB/2.5kB (12.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 323.6kB/323.6kB (877.8kB/s) raw.png Done... Downloading [####################]100.00% 75.4MB/75.4MB (3.0MB/s) 10s_video.zip.synapse_download_40878256 Done... Downloading [####################]100.00% 3.6kB/3.6kB (11.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 96.6MB/96.6MB (3.0MB/s) 10s_video.zip.synapse_download_40878264 Done... Downloading [####################]100.00% 452.6kB/452.6kB (829.0kB/s) raw.png Done... Downloading [####################]100.00% 3.5kB/3.5kB (7.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 101.0MB/101.0MB (4.5MB/s) 10s_video.zip.synapse_download_40878227 Done... Downloading [####################]100.00% 371.3kB/371.3kB (1013.9kB/s) raw.png Done... Downloading [####################]100.00% 1.9kB/1.9kB (6.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 347.4kB/347.4kB (944.3kB/s) raw.png Done... Downloading [####################]100.00% 93.9MB/93.9MB (2.9MB/s) 10s_video.zip.synapse_download_40878209 Done... Downloading [####################]100.00% 83.4MB/83.4MB (4.6MB/s) 10s_video.zip.synapse_download_40878246 Done... Downloading [####################]100.00% 96.6MB/96.6MB (5.8MB/s) 10s_video.zip.synapse_download_40878214 Done... Downloading [####################]100.00% 1.5kB/1.5kB (6.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 1.6kB/1.6kB (3.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 354.1kB/354.1kB (542.4kB/s) raw.png Done... Downloading [####################]100.00% 338.9kB/338.9kB (708.7kB/s) raw.png Done... Downloading [####################]100.00% 1.6kB/1.6kB (4.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 291.2kB/291.2kB (794.4kB/s) raw.png Done... Downloading [####################]100.00% 96.5MB/96.5MB (3.1MB/s) 10s_video.zip.synapse_download_40878222 Done... Downloading [####################]100.00% 1.9kB/1.9kB (9.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 379.2kB/379.2kB (757.4kB/s) raw.png Done... Downloading [####################]100.00% 93.0MB/93.0MB (2.5MB/s) 10s_video.zip.synapse_download_40878217 Done... Downloading [####################]100.00% 1.8kB/1.8kB (6.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 380.2kB/380.2kB (830.7kB/s) raw.png Done... Downloading [####################]100.00% 95.7MB/95.7MB (5.5MB/s) 10s_video.zip.synapse_download_40878165 Done... Downloading [####################]100.00% 1.3kB/1.3kB (4.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 255.2kB/255.2kB (935.3kB/s) raw.png Done... Downloading [####################]100.00% 2.1kB/2.1kB (6.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 89.5MB/89.5MB (2.0MB/s) 10s_video.zip.synapse_download_40878216 Done... Downloading [####################]100.00% 367.1kB/367.1kB (3.9MB/s) raw.png Done... Downloading [####################]100.00% 69.8MB/69.8MB (2.7MB/s) 10s_video.zip.synapse_download_40878207 Done... Downloading [####################]100.00% 89.9MB/89.9MB (3.0MB/s) 10s_video.zip.synapse_download_40878210 Done... Downloading [####################]100.00% 78.8MB/78.8MB (3.9MB/s) 10s_video.zip.synapse_download_40878164 Done... Downloading [####################]100.00% 95.7MB/95.7MB (3.8MB/s) 10s_video.zip.synapse_download_40878170 Done... \ Downloaded 17.3GB (18.8MB/s)Traceback (most recent call last): File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseutils/sync.py", line 255, in _sync_file entity = self._syn.get( File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseclient/client.py", line 665, in get return self._getWithEntityBundle(entityBundle=bundle, entity=entity, **kwargs) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseclient/client.py", line 781, in _getWithEntityBundle self._download_file_entity(downloadLocation, entity, ifcollision, submission) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseclient/client.py", line 804, in _download_file_entity cached_file_path = self.cache.get(entity.dataFileHandleId, downloadLocation) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseclient/core/cache.py", line 159, in get cache_map = self._read_cache_map(cache_dir) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseclient/core/cache.py", line 106, in _read_cache_map cache_map = json.load(f) File "/usr/lib/python3.8/json/__init__.py", line 293, in load return loads(fp.read(), File "/usr/lib/python3.8/json/__init__.py", line 357, in loads return _default_decoder.decode(s) File "/usr/lib/python3.8/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/guys/.virtualenvs/endovis2019/bin/synapse", line 33, in
sys.exit(load_entry_point('synapseclient==2.2.2', 'console_scripts', 'synapse')()) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseclient/__main__.py", line 902, in main perform_main(args, syn) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseclient/__main__.py", line 852, in perform_main args.func(args, syn) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseclient/__main__.py", line 77, in get synapseutils.syncFromSynapse(syn, args.id, args.downloadLocation, followLink=args.followLink) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseutils/sync.py", line 96, in syncFromSynapse files = sync_from_synapse.sync(entity, path, ifcollision, followLink) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseutils/sync.py", line 228, in sync root_folder_sync = self._sync_root(entity, path, ifcollision, followLink, progress) File "/home/guys/.virtualenvs/endovis2019/lib/python3.8/site-packages/synapseutils/sync.py", line 305, in _sync_root raise ValueError("File download failed during sync") from exception ValueError: File download failed during sync
I let the full log for this download in case it might help. If you have any idea what could be the reason of the problem. Thanks in advance,
@AnnikaReinke Thanks a lot for your quick answer. I thought that since the challenge is already finished, I didn't have to register to get the data. But ok? I'll do that, thanks!
Dear @Mordokkai, only users registered to the challenge are able to download the data. Please follow the instructions provided in the challenge's wiki. Best wishes, Annika Reinke
Hi all, I'm getting the same error as @aashu when I try to download the data: [WARNING] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING: You have READ permission on this file entity but not DOWNLOAD permission. The file has NOT been downloaded. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I created an account on synapse and checked the project_id = "syn20575265". I don't know why I can't get access to the data. Do I have to ask someone to get permission ? @AnnikaReinke @schnobi1990 Thanks in advance
@jordank Hi, the command doesn't work, I'm gonna just check it frequently. Thank you very much.
@SIAT-HCI some errors above show that connections are being refused from your network (for example the curl command connection above was refused and this is not related to Synapse). Are you running behind a proxy or firewall that might be interfering with outbound connections? There also seem to be some connection issues connecting to AWS where the Synapse data is stored. Can you add the following command to the top of your download.py? If it's true that your environment may be interfering with outbound connections it's possible that it might it might allow one connection through at a time. ``` import synapseclient.core.config synapseclient.core.config.single_threaded = True ```
@jordank Hi, the running results of "curl" and "synapse" are showed as follows, the download process is stuck at "12 GB" for about 7 hours, the downloading logs of the begining and end are pasted, I think it would be enough to analyze. Thanks a lot. ``` curl https://raw.githubusercontent.com/Sage-Bionetworks/synapsePythonClie nt/master/synapseclient/synapsePythonClient curl: (7) Failed to connect to raw.githubusercontent.com port 443: Connection refused ****************************************************************************************************************************************** Space Bar ****************************************************************************************************************************************** $ synapse --debug get -r syn20575265 --downloadLocation "/home/licaizi/DataSet/Robust Medical Instrument Segmentation(2019)" Exception in version check: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /Sage-Bionetworks/synapsePythonClient/master/synapseclient/synapsePythonClient (Caused by NewConnectionError('
: Failed to establish a new connection: [Errno 111] Connection refused',)) 2020-11-04 10:29:08,426 [retry:61 - DEBUG]: retrying on status code: 429 2020-11-04 10:29:08,426 [retry:63 - DEBUG]: Too many concurrent requests. Allowed 3 concurrent connections at any time. 2020-11-04 10:29:08,426 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 1.0 seconds... 2020-11-04 10:30:29,120 [retry:61 - DEBUG]: retrying on status code: 429 2020-11-04 10:30:29,120 [retry:63 - DEBUG]: Too many concurrent requests. Allowed 3 concurrent connections at any time. 2020-11-04 10:30:29,120 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 0.8 seconds... 2020-11-04 10:30:58,915 [retry:61 - DEBUG]: retrying on status code: 429 2020-11-04 10:30:58,915 [retry:63 - DEBUG]: Too many concurrent requests. Allowed 3 concurrent connections at any time. 2020-11-04 10:30:58,915 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 0.9 seconds... 2020-11-04 10:31:19,965 [retry:61 - DEBUG]: retrying on status code: 429 2020-11-04 10:31:19,965 [retry:63 - DEBUG]: Too many concurrent requests. Allowed 3 concurrent connections at any time. 2020-11-04 10:31:19,966 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 0.9 seconds... 2020-11-04 10:31:57,540 [retry:61 - DEBUG]: retrying on status code: 429 2020-11-04 10:31:57,540 [retry:63 - DEBUG]: Too many concurrent requests. Allowed 3 concurrent connections at any time. 2020-11-04 10:31:57,540 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 1.4 seconds... 2020-11-04 10:32:18,448 [retry:61 - DEBUG]: retrying on status code: 429 2020-11-04 10:32:18,448 [retry:63 - DEBUG]: Too many concurrent requests. Allowed 3 concurrent connections at any time. 2020-11-04 10:32:18,448 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 1.4 seconds... Downloading [####################]100.00% 91.9MB/91.9MB (673.7kB/s) 10s_video.zip.synapse_download_40887631 Done... Downloading [####################]100.00% 2.1kB/2.1kB (874.4kB/s) instrument_instances.png Done... Downloading [####################]100.00% 426.0kB/426.0kB (224.2kB/s) raw.png Done... Downloading [####################]100.00% 94.8MB/94.8MB (603.6kB/s) 10s_video.zip.synapse_download_40887625 Done... Downloading [####################]100.00% 83.9MB/83.9MB (527.6kB/s) 10s_video.zip.synapse_download_40887660 Done... Downloading [####################]100.00% 101.5MB/101.5MB (701.2kB/s) 10s_video.zip.synapse_download_42125203 Done... Downloading [####################]100.00% 2.5kB/2.5kB (1.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 2.3kB/2.3kB (6.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 97.1MB/97.1MB (595.2kB/s) 10s_video.zip.synapse_download_42125224 Done... Downloading [####################]100.00% 349.0kB/349.0kB (274.5kB/s) raw.png Done... Downloading [####################]100.00% 2.1kB/2.1kB (4.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 421.3kB/421.3kB (76.1kB/s) raw.png Done... Downloading [####################]100.00% 411.9kB/411.9kB (784.0kB/s) raw.png Done... Downloading [####################]100.00% 2.3kB/2.3kB (4.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 429.5kB/429.5kB (562.1kB/s) raw.png Done... Downloading [####################]100.00% 100.0MB/100.0MB (523.5kB/s) 10s_video.zip.synapse_download_42125220 Done... Downloading [####################]100.00% 2.5kB/2.5kB (2.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 389.2kB/389.2kB (330.7kB/s) raw.png Done... Downloading [####################]100.00% 100.6MB/100.6MB (933.7kB/s) 10s_video.zip.synapse_download_40887581 Done... Downloading [####################]100.00% 3.6kB/3.6kB (6.9MB/s) instrument_instances.png Done... - Downloaded 907.4MB (1.5MB/s)2020-11-04 10:37:50,988 [retry:52 - DEBUG]: calling
.
at 0x7fd824c96488> resulted in an Exception 2020-11-04 10:37:50,988 [retry:86 - DEBUG]: retrying exception: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2020-11-04 10:37:50,988 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 0.8 seconds... / Downloaded 931.4MB (1.5MB/s)2020-11-04 10:38:03,720 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 276, in download_file file_size = _get_file_size(url_info.url) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 203, in _get_file_size return int(res_get.headers['Content-Length']) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/structures.py", line 54, in __getitem__ return self._store[key.lower()][1] KeyError: 'content-length' Downloading [####################]100.00% 91.2MB/91.2MB (578.8kB/s) 10s_video.zip.synapse_download_40887577 Done... Downloading [####################]100.00% 93.0MB/93.0MB (748.7kB/s) 10s_video.zip.synapse_download_40887562 Done... Downloading [####################]100.00% 103.4MB/103.4MB (544.3kB/s) 10s_video.zip.synapse_download_40887603 Done... Downloading [####################]100.00% 386.8kB/386.8kB (373.0kB/s) raw.png Done... Downloading [####################]100.00% 102.5MB/102.5MB (514.9kB/s) 10s_video.zip.synapse_download_40887616 Done... Downloading [####################]100.00% 1.4kB/1.4kB (1.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 431.9kB/431.9kB (816.8kB/s) raw.png Done... Downloading [####################]100.00% 80.2MB/80.2MB (733.4kB/s) 10s_video.zip.synapse_download_40887571 Done... Downloading [####################]100.00% 101.7MB/101.7MB (669.0kB/s) 10s_video.zip.synapse_download_40887610 Done... \ Downloaded 1.4GB (1.8MB/s)2020-11-04 10:40:45,447 [retry:52 - DEBUG]: calling
.
at 0x7fd824cf3048> resulted in an Exception 2020-11-04 10:40:45,447 [retry:86 - DEBUG]: retrying exception: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/3384422/61b02558-ae4d-4444-a7cf-60ec95ed3d49/instrument_instances.png?response-content-disposition=attachment%3B%20filename%3D%22instrument_instances.png%22%3B%20filename%2A%3Dutf-8%27%27instrument%255Finstances.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHMaCXVzLWVhc3QtMSJHMEUCIQCflFKj6orsMbx8HvfVCyQpRpvAPaCDUUB5As7HznMAbwIgeOF%2BYhU9GacT78NOq96Gkl39S6bopWUSHA%2BBhUE2g4YqvQMIy%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwzMjU1NjU1ODU4MzkiDMpLWfaz%2B6%2B6M8yCiiqRA2zNKE%2FhSFhV5pcPPukW0sDGdRPiMuMacXb51mYKSrZDDCIpaU7iHwRjcvWkDfWoGXZLm1icje4X9qv6iFnbCmwCniNyAiRUvhwuqKOCMIC8LlpcYDiUxvG92Zg3%2FZRf6H35yLJ9OfN0zWLErURp46FAzhKAgJUoA2dVdNexR%2BVsIG00%2F%2FugRlVjo83xOtovODFTqaxKXyVCU4BHU934qegBIreyadG%2F373GCg7o47b3o6yePCmHMxrRh%2Ba%2FleT%2BbDxnYNfqQvMtVXMyErr52iW9Y1mK%2Fpd1aj2168lQLzHunkcZ7Lar8YM0FS0D9P10ZzVnPJcvNlBJWjmfL0w3VjcxOaVBOtMuJHE3hjFmoThRKAehxiYCifmTmB0GWi1RfIaWtZ1%2BCiCGcBL1I1DqQsqgZaNBlUoqwOgWIZgBmlljUtoJhWUwia8ftXwS1IJMKItLeQVSD8tqUKFzkucbCPmyubr9HKcAgsVfRuoWRiv1m2zGBYFp%2BIYPl4BCmYc3YwRn%2B7Oj3Sdyad%2FGZgTxpO2FMIKaiP0FOusBJJpjjI1jt%2FTRj%2FtDxiCAJWXLmjnZMI1nwY%2BvqqfzZEo%2B3lGgWHXwbaraPyYtu3YF%2BFN%2B0Ree%2FjQqBv0fRoGB6lCf%2Bkzcl76TDSMr4sj1K%2Br%2BY5L8O0PqjHCB%2BqIvUUzHcKeYLXRblDkhp5EV8%2FoHdZYS2ZhSlG80Ry3GZEnmkIfVng5Uy23OUSBb2MjpHWhb1sLr075YDs9iAS6VbFM9AUfcab5pGR4at263zTcOVL1l46CtXuZl3qrEEOAM2MMHQK8%2BzrGAQtF8jR3FdiGUDG3%2B7ci8HtQVP7PhTQcVPQzjxRbEtQmgZ%2BAVWA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T023836Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGXZI3TIWO4%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=3f20c83f3c736f8c0e0c47ded7af6a767315076695bec6bf3d13d17b8cedc035 (Caused by NewConnectionError('
: Failed to establish a new connection: [Errno 110] Connection timed out',)) 2020-11-04 10:40:45,447 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 1.4 seconds... | Downloaded 1.4GB (1.8MB/s)2020-11-04 10:40:48,141 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1798, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1900, in _download_from_URL exceptions._raise_for_status(response, verbose=self.debug) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/exceptions.py", line 160, in _raise_for_status raise SynapseHTTPError(message, response=response) synapseclient.core.exceptions.SynapseHTTPError: 403 Client Error: Forbidden
AccessDenied
Request has expired
30
2020-11-04T02:39:06Z
2020-11-04T02:40:48Z
D502995865B09681
H822AFfVeUZz41xJDAfkcmJUB9J39bqAOg3r21qJKQNDvJkI1LlXfYGIoxmAKvjjGHA9RMg/fQU=
>>>>>> Request <<<<<< https://s3.amazonaws.com/proddata.sagebase.org/3384422/61b02558-ae4d-4444-a7cf-60ec95ed3d49/instrument_instances.png?response-content-disposition=attachment%3B%20filename%3D%22instrument_instances.png%22%3B%20filename%2A%3Dutf-8%27%27instrument%255Finstances.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHMaCXVzLWVhc3QtMSJHMEUCIQCflFKj6orsMbx8HvfVCyQpRpvAPaCDUUB5As7HznMAbwIgeOF%2BYhU9GacT78NOq96Gkl39S6bopWUSHA%2BBhUE2g4YqvQMIy%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwzMjU1NjU1ODU4MzkiDMpLWfaz%2B6%2B6M8yCiiqRA2zNKE%2FhSFhV5pcPPukW0sDGdRPiMuMacXb51mYKSrZDDCIpaU7iHwRjcvWkDfWoGXZLm1icje4X9qv6iFnbCmwCniNyAiRUvhwuqKOCMIC8LlpcYDiUxvG92Zg3%2FZRf6H35yLJ9OfN0zWLErURp46FAzhKAgJUoA2dVdNexR%2BVsIG00%2F%2FugRlVjo83xOtovODFTqaxKXyVCU4BHU934qegBIreyadG%2F373GCg7o47b3o6yePCmHMxrRh%2Ba%2FleT%2BbDxnYNfqQvMtVXMyErr52iW9Y1mK%2Fpd1aj2168lQLzHunkcZ7Lar8YM0FS0D9P10ZzVnPJcvNlBJWjmfL0w3VjcxOaVBOtMuJHE3hjFmoThRKAehxiYCifmTmB0GWi1RfIaWtZ1%2BCiCGcBL1I1DqQsqgZaNBlUoqwOgWIZgBmlljUtoJhWUwia8ftXwS1IJMKItLeQVSD8tqUKFzkucbCPmyubr9HKcAgsVfRuoWRiv1m2zGBYFp%2BIYPl4BCmYc3YwRn%2B7Oj3Sdyad%2FGZgTxpO2FMIKaiP0FOusBJJpjjI1jt%2FTRj%2FtDxiCAJWXLmjnZMI1nwY%2BvqqfzZEo%2B3lGgWHXwbaraPyYtu3YF%2BFN%2B0Ree%2FjQqBv0fRoGB6lCf%2Bkzcl76TDSMr4sj1K%2Br%2BY5L8O0PqjHCB%2BqIvUUzHcKeYLXRblDkhp5EV8%2FoHdZYS2ZhSlG80Ry3GZEnmkIfVng5Uy23OUSBb2MjpHWhb1sLr075YDs9iAS6VbFM9AUfcab5pGR4at263zTcOVL1l46CtXuZl3qrEEOAM2MMHQK8%2BzrGAQtF8jR3FdiGUDG3%2B7ci8HtQVP7PhTQcVPQzjxRbEtQmgZ%2BAVWA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T023836Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGXZI3TIWO4%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=3f20c83f3c736f8c0e0c47ded7af6a767315076695bec6bf3d13d17b8cedc035 GET >>> Headers: {'User-Agent': 'synapsecommandlineclient synapseclient/2.2.2 python-requests/2.24.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'userId': 'cz.li@siat.ac.cn', 'signatureTimestamp': '2020-11-04T02:40:46.000Z', 'signature': b'uJ3Ft9RmwlsGw3dkfkIJ50Ee6eA='} >>> Body: None >>>>>> Response <<<<<<
>>> Headers: {'x-amz-request-id': 'D502995865B09681', 'x-amz-id-2': 'H822AFfVeUZz41xJDAfkcmJUB9J39bqAOg3r21qJKQNDvJkI1LlXfYGIoxmAKvjjGHA9RMg/fQU=', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Date': 'Wed, 04 Nov 2020 02:40:47 GMT', 'Server': 'AmazonS3'} >>> Body:
AccessDenied
Request has expired
30
2020-11-04T02:39:06Z
2020-11-04T02:40:48Z
D502995865B09681
H822AFfVeUZz41xJDAfkcmJUB9J39bqAOg3r21qJKQNDvJkI1LlXfYGIoxmAKvjjGHA9RMg/fQU=
Downloading [####################]100.00% 317.9kB/317.9kB (77.7kB/s) raw.png Done... Downloading [####################]100.00% 101.3MB/101.3MB (875.4kB/s) 10s_video.zip.synapse_download_40887586 Done... Downloading [####################]100.00% 1.8kB/1.8kB (3.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 207.0kB/207.0kB (54.4kB/s) raw.png Done... Downloading [####################]100.00% 1.7kB/1.7kB (3.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 402.1kB/402.1kB (73.8kB/s) raw.png Done... Downloading [####################]100.00% 1.5kB/1.5kB (3.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 419.5kB/419.5kB (362.4kB/s) raw.png Done... Downloading [####################]100.00% 94.4MB/94.4MB (579.1kB/s) 10s_video.zip.synapse_download_42125200 Done... Downloading [####################]100.00% 1010.0bytes/1010.0bytes (1.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 424.1kB/424.1kB (372.0kB/s) raw.png Done... / Downloaded 1.7GB (1.9MB/s)2020-11-04 10:43:08,003 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 436, in _error_catcher yield File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 518, in read data = self._fp.read(amt) if not fp_closed else b"" File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/http/client.py", line 459, in read n = self.readinto(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/http/client.py", line 503, in readinto n = self.fp.readinto(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 1012, in recv_into return self.read(nbytes, buffer) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 874, in read return self._sslobj.read(len, buffer) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 631, in read v = self._sslobj.read(len, buffer) ConnectionResetError: [Errno 104] Connection reset by peer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 751, in generate for chunk in self.raw.stream(chunk_size, decode_content=True): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 575, in stream data = self.read(amt=amt, decode_content=decode_content) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 540, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/contextlib.py", line 99, in __exit__ self.gen.throw(type, value, traceback) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 454, in _error_catcher raise ProtocolError("Connection broken: %r" % e, e) urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1798, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1947, in _download_from_URL for nChunks, chunk in enumerate(response.iter_content(FILE_BUFFER_SIZE)): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 754, in generate raise ChunkedEncodingError(e) requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) Downloading [####################]100.00% 375.4kB/375.4kB (352.8kB/s) raw.png Done... Downloading [####################]100.00% 96.4MB/96.4MB (814.8kB/s) 10s_video.zip.synapse_download_42125196 Done... Downloading [####################]100.00% 1.8kB/1.8kB (3.5MB/s) instrument_instances.png Done... Downloading [####################]100.00% 418.9kB/418.9kB (529.9kB/s) raw.png Done... Downloading [####################]100.00% 1.2kB/1.2kB (3.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 389.8kB/389.8kB (716.0kB/s) raw.png Done... Downloading [####################]100.00% 96.4MB/96.4MB (768.5kB/s) 10s_video.zip.synapse_download_42125178 Done... Downloading [####################]100.00% 1.7kB/1.7kB (3.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 393.2kB/393.2kB (136.2kB/s) raw.png Done... Downloading [####################]100.00% 95.8MB/95.8MB (584.3kB/s) 10s_video.zip.synapse_download_42125182 Done... - Downloaded 1.9GB (2.0MB/s)2020-11-04 10:44:10,378 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 276, in download_file file_size = _get_file_size(url_info.url) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 203, in _get_file_size return int(res_get.headers['Content-Length']) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/structures.py", line 54, in __getitem__ return self._store[key.lower()][1] KeyError: 'content-length' Downloading [####################]100.00% 1.4kB/1.4kB (4.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 404.4kB/404.4kB (336.3kB/s) raw.png Done... Downloading [####################]100.00% 93.6MB/93.6MB (902.6kB/s) 10s_video.zip.synapse_download_40887580 Done... Downloading [####################]100.00% 1.6kB/1.6kB (3.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 94.8MB/94.8MB (603.8kB/s) 10s_video.zip.synapse_download_42125173 Done... Downloading [####################]100.00% 407.4kB/407.4kB (249.5kB/s) raw.png Done... Downloading [####################]100.00% 962.0bytes/962.0bytes (1.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 355.5kB/355.5kB (653.1kB/s) raw.png Done... Downloading [####################]100.00% 92.7MB/92.7MB (717.8kB/s) 10s_video.zip.synapse_download_42125185 Done... Downloading [####################]100.00% 2.0kB/2.0kB (3.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 352.4kB/352.4kB (329.6kB/s) raw.png Done... Downloading [####################]100.00% 92.0MB/92.0MB (543.5kB/s) 10s_video.zip.synapse_download_40887565 Done... Downloading [####################]100.00% 1.9kB/1.9kB (799.1kB/s) instrument_instances.png Done... Downloading [####################]100.00% 93.3MB/93.3MB (668.4kB/s) 10s_video.zip.synapse_download_42125165 Done... Downloading [####################]100.00% 346.1kB/346.1kB (294.8kB/s) raw.png Done... Downloading [####################]100.00% 1.9kB/1.9kB (4.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 426.2kB/426.2kB (720.4kB/s) raw.png Done... Downloading [####################]100.00% 90.9MB/90.9MB (722.8kB/s) 10s_video.zip.synapse_download_40887594 Done... Downloading [####################]100.00% 90.9MB/90.9MB (621.7kB/s) 10s_video.zip.synapse_download_42125140 Done... \ Downloaded 2.5GB (2.1MB/s)2020-11-04 10:47:31,764 [retry:52 - DEBUG]: calling
.
at 0x7fd824c96730> resulted in an Exception 2020-11-04 10:47:31,764 [retry:101 - DEBUG]: retries have run out. re-raising the exception NoneType: None 2020-11-04 10:47:31,764 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request self._validate_conn(conn) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn conn.connect() File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect ssl_context=context, File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 384, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 407, in wrap_socket _context=self, _session=session) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 817, in __init__ self.do_handshake() File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 1077, in do_handshake self._sslobj.do_handshake() File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 689, in do_handshake self._sslobj.do_handshake() ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:852) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/util/retry.py", line 439, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/3384422/3253f73e-ba34-4962-a58f-a1456c2d2302/instrument_instances.png?response-content-disposition=attachment%3B%20filename%3D%22instrument_instances.png%22%3B%20filename%2A%3Dutf-8%27%27instrument%255Finstances.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHMaCXVzLWVhc3QtMSJHMEUCIQCflFKj6orsMbx8HvfVCyQpRpvAPaCDUUB5As7HznMAbwIgeOF%2BYhU9GacT78NOq96Gkl39S6bopWUSHA%2BBhUE2g4YqvQMIy%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwzMjU1NjU1ODU4MzkiDMpLWfaz%2B6%2B6M8yCiiqRA2zNKE%2FhSFhV5pcPPukW0sDGdRPiMuMacXb51mYKSrZDDCIpaU7iHwRjcvWkDfWoGXZLm1icje4X9qv6iFnbCmwCniNyAiRUvhwuqKOCMIC8LlpcYDiUxvG92Zg3%2FZRf6H35yLJ9OfN0zWLErURp46FAzhKAgJUoA2dVdNexR%2BVsIG00%2F%2FugRlVjo83xOtovODFTqaxKXyVCU4BHU934qegBIreyadG%2F373GCg7o47b3o6yePCmHMxrRh%2Ba%2FleT%2BbDxnYNfqQvMtVXMyErr52iW9Y1mK%2Fpd1aj2168lQLzHunkcZ7Lar8YM0FS0D9P10ZzVnPJcvNlBJWjmfL0w3VjcxOaVBOtMuJHE3hjFmoThRKAehxiYCifmTmB0GWi1RfIaWtZ1%2BCiCGcBL1I1DqQsqgZaNBlUoqwOgWIZgBmlljUtoJhWUwia8ftXwS1IJMKItLeQVSD8tqUKFzkucbCPmyubr9HKcAgsVfRuoWRiv1m2zGBYFp%2BIYPl4BCmYc3YwRn%2B7Oj3Sdyad%2FGZgTxpO2FMIKaiP0FOusBJJpjjI1jt%2FTRj%2FtDxiCAJWXLmjnZMI1nwY%2BvqqfzZEo%2B3lGgWHXwbaraPyYtu3YF%2BFN%2B0Ree%2FjQqBv0fRoGB6lCf%2Bkzcl76TDSMr4sj1K%2Br%2BY5L8O0PqjHCB%2BqIvUUzHcKeYLXRblDkhp5EV8%2FoHdZYS2ZhSlG80Ry3GZEnmkIfVng5Uy23OUSBb2MjpHWhb1sLr075YDs9iAS6VbFM9AUfcab5pGR4at263zTcOVL1l46CtXuZl3qrEEOAM2MMHQK8%2BzrGAQtF8jR3FdiGUDG3%2B7ci8HtQVP7PhTQcVPQzjxRbEtQmgZ%2BAVWA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T024659Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGXZI3TIWO4%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=a8d7eb79ad8c5f9b44354b6e8f3783eb5ce963bd60e628f432a0989c420fe6c3 (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:852)'),)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1798, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1898, in _download_from_URL verbose=self.debug, **STANDARD_RETRY_PARAMS) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/retry.py", line 102, in with_retry raise exc File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/retry.py", line 48, in with_retry response = function() File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1897, in
stream=True, allow_redirects=False), File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/sessions.py", line 543, in get return self.request('GET', url, **kwargs) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/adapters.py", line 514, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/3384422/3253f73e-ba34-4962-a58f-a1456c2d2302/instrument_instances.png?response-content-disposition=attachment%3B%20filename%3D%22instrument_instances.png%22%3B%20filename%2A%3Dutf-8%27%27instrument%255Finstances.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHMaCXVzLWVhc3QtMSJHMEUCIQCflFKj6orsMbx8HvfVCyQpRpvAPaCDUUB5As7HznMAbwIgeOF%2BYhU9GacT78NOq96Gkl39S6bopWUSHA%2BBhUE2g4YqvQMIy%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwzMjU1NjU1ODU4MzkiDMpLWfaz%2B6%2B6M8yCiiqRA2zNKE%2FhSFhV5pcPPukW0sDGdRPiMuMacXb51mYKSrZDDCIpaU7iHwRjcvWkDfWoGXZLm1icje4X9qv6iFnbCmwCniNyAiRUvhwuqKOCMIC8LlpcYDiUxvG92Zg3%2FZRf6H35yLJ9OfN0zWLErURp46FAzhKAgJUoA2dVdNexR%2BVsIG00%2F%2FugRlVjo83xOtovODFTqaxKXyVCU4BHU934qegBIreyadG%2F373GCg7o47b3o6yePCmHMxrRh%2Ba%2FleT%2BbDxnYNfqQvMtVXMyErr52iW9Y1mK%2Fpd1aj2168lQLzHunkcZ7Lar8YM0FS0D9P10ZzVnPJcvNlBJWjmfL0w3VjcxOaVBOtMuJHE3hjFmoThRKAehxiYCifmTmB0GWi1RfIaWtZ1%2BCiCGcBL1I1DqQsqgZaNBlUoqwOgWIZgBmlljUtoJhWUwia8ftXwS1IJMKItLeQVSD8tqUKFzkucbCPmyubr9HKcAgsVfRuoWRiv1m2zGBYFp%2BIYPl4BCmYc3YwRn%2B7Oj3Sdyad%2FGZgTxpO2FMIKaiP0FOusBJJpjjI1jt%2FTRj%2FtDxiCAJWXLmjnZMI1nwY%2BvqqfzZEo%2B3lGgWHXwbaraPyYtu3YF%2BFN%2B0Ree%2FjQqBv0fRoGB6lCf%2Bkzcl76TDSMr4sj1K%2Br%2BY5L8O0PqjHCB%2BqIvUUzHcKeYLXRblDkhp5EV8%2FoHdZYS2ZhSlG80Ry3GZEnmkIfVng5Uy23OUSBb2MjpHWhb1sLr075YDs9iAS6VbFM9AUfcab5pGR4at263zTcOVL1l46CtXuZl3qrEEOAM2MMHQK8%2BzrGAQtF8jR3FdiGUDG3%2B7ci8HtQVP7PhTQcVPQzjxRbEtQmgZ%2BAVWA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T024659Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGXZI3TIWO4%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=a8d7eb79ad8c5f9b44354b6e8f3783eb5ce963bd60e628f432a0989c420fe6c3 (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:852)'),)) - Downloaded 2.5GB (2.2MB/s)2020-11-04 10:47:57,742 [retry:52 - DEBUG]: calling
.
at 0x7fd824cbc378> resulted in an Exception 2020-11-04 10:47:57,742 [retry:86 - DEBUG]: retrying exception: ('Connection aborted.', OSError(0, 'Error')) 2020-11-04 10:47:57,742 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 1.1 seconds... \ Downloaded 2.6GB (2.2MB/s)2020-11-04 10:48:00,700 [retry:52 - DEBUG]: calling
.
at 0x7fd840e589d8> resulted in an Exception 2020-11-04 10:48:00,700 [retry:101 - DEBUG]: retries have run out. re-raising the exception NoneType: None 2020-11-04 10:48:00,701 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request self._validate_conn(conn) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn conn.connect() File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect ssl_context=context, File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 384, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 407, in wrap_socket _context=self, _session=session) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 817, in __init__ self.do_handshake() File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 1077, in do_handshake self._sslobj.do_handshake() File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 689, in do_handshake self._sslobj.do_handshake() ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:852) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/util/retry.py", line 439, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/3384422/3253f73e-ba34-4962-a58f-a1456c2d2302/instrument_instances.png?response-content-disposition=attachment%3B%20filename%3D%22instrument_instances.png%22%3B%20filename%2A%3Dutf-8%27%27instrument%255Finstances.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHIaCXVzLWVhc3QtMSJIMEYCIQChpk2d13vOsZEepY6%2B0iozS0IhWoo1H9SH59AtelpxxQIhALYnUYhKaZVtFdcjkpYxrRg%2BAEyXyy4ZH3ihU4Xn44D%2FKr0DCMv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQABoMMzI1NTY1NTg1ODM5IgxGHN7Dk8R%2B85ve4ToqkQNO2hLlUNPo5lpKfs1fo65tfP433QcSUGSm2iR%2FFADkPuCQdbL7%2BKLTj4FbEccKnIWp9RFvCSgtA46ZMKlBsWkyWl32AKwSyKgkrdFkgb4SW8cmf%2FdZWyBD%2F5iAWZIsH8%2BuafTzAyRU5kA62ZZt8TtTeTRqkAaLyETt%2F1AQQIINyK48EiW5ucsNiHV%2BX7r5MGn7a0lW%2FNgGxS0emtF0yUgbCCcpgHuh28enZ1XLBetPccJake7oRy%2Fy8SrbFoid6S9HTXt%2BTF5LeSeKWRrgZQe5j5UYJnv0FJxkvvXL65Rw4CEZxID%2FKe473zQPHrxKFKiJKRzKVUef4L4G%2FL0XP9%2FoE0GBzkJFC2lRCnVBSvz%2BLAiSH%2B0btS7RdOlaFSlYyFu3ueomONriFH019y4752CQ08l1%2BdfgMR8V3HwoyMYZWLvwVxNPErULsWMZccGhq6aNAgea5i5flJLvqUFV75dvF8H0PWF1aQ78k0wUQ5whIXx%2FRqANN6qCdetulCNY0pu%2ByTLE64f5v1j496dCsmweoDDejoj9BTrqAex0riIlIE9EAjrXI1ITlE7WO7fq1nyaGbmkT34RcLi1rARFluILzi0Yll8DzMLc8Z1ANmKjlwF9M%2F0pYEv9vrihyPkqanaP2ifYteVzcKVaQi7KYJDcrwqULKqnW5V7OzLZ4kKJ8TVqvr1i8dicz4raIo51AwI%2FoOjSfin6K6646%2F8%2FrgU2ZNVjrKwvsDZwNP%2FSw8G9PwtmZH%2BOSwfXWLqYdJUEBk5%2BFNs%2B0Mr6F8FDFI3jBGupvNHlsPOEO%2FiiUAPICjIwmQGmevQu31%2Fmpf93noxyUxZ8JsaRQPrgHs8EqBABjk20MixJmQ%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T024731Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGX5ZM32TPU%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=bfef91296fc6f44533d240851b59db9486f50c660c02cfc54cc6c1294554986a (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:852)'),)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1798, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1898, in _download_from_URL verbose=self.debug, **STANDARD_RETRY_PARAMS) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/retry.py", line 102, in with_retry raise exc File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/retry.py", line 48, in with_retry response = function() File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1897, in
stream=True, allow_redirects=False), File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/sessions.py", line 543, in get return self.request('GET', url, **kwargs) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/adapters.py", line 514, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/3384422/3253f73e-ba34-4962-a58f-a1456c2d2302/instrument_instances.png?response-content-disposition=attachment%3B%20filename%3D%22instrument_instances.png%22%3B%20filename%2A%3Dutf-8%27%27instrument%255Finstances.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHIaCXVzLWVhc3QtMSJIMEYCIQChpk2d13vOsZEepY6%2B0iozS0IhWoo1H9SH59AtelpxxQIhALYnUYhKaZVtFdcjkpYxrRg%2BAEyXyy4ZH3ihU4Xn44D%2FKr0DCMv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQABoMMzI1NTY1NTg1ODM5IgxGHN7Dk8R%2B85ve4ToqkQNO2hLlUNPo5lpKfs1fo65tfP433QcSUGSm2iR%2FFADkPuCQdbL7%2BKLTj4FbEccKnIWp9RFvCSgtA46ZMKlBsWkyWl32AKwSyKgkrdFkgb4SW8cmf%2FdZWyBD%2F5iAWZIsH8%2BuafTzAyRU5kA62ZZt8TtTeTRqkAaLyETt%2F1AQQIINyK48EiW5ucsNiHV%2BX7r5MGn7a0lW%2FNgGxS0emtF0yUgbCCcpgHuh28enZ1XLBetPccJake7oRy%2Fy8SrbFoid6S9HTXt%2BTF5LeSeKWRrgZQe5j5UYJnv0FJxkvvXL65Rw4CEZxID%2FKe473zQPHrxKFKiJKRzKVUef4L4G%2FL0XP9%2FoE0GBzkJFC2lRCnVBSvz%2BLAiSH%2B0btS7RdOlaFSlYyFu3ueomONriFH019y4752CQ08l1%2BdfgMR8V3HwoyMYZWLvwVxNPErULsWMZccGhq6aNAgea5i5flJLvqUFV75dvF8H0PWF1aQ78k0wUQ5whIXx%2FRqANN6qCdetulCNY0pu%2ByTLE64f5v1j496dCsmweoDDejoj9BTrqAex0riIlIE9EAjrXI1ITlE7WO7fq1nyaGbmkT34RcLi1rARFluILzi0Yll8DzMLc8Z1ANmKjlwF9M%2F0pYEv9vrihyPkqanaP2ifYteVzcKVaQi7KYJDcrwqULKqnW5V7OzLZ4kKJ8TVqvr1i8dicz4raIo51AwI%2FoOjSfin6K6646%2F8%2FrgU2ZNVjrKwvsDZwNP%2FSw8G9PwtmZH%2BOSwfXWLqYdJUEBk5%2BFNs%2B0Mr6F8FDFI3jBGupvNHlsPOEO%2FiiUAPICjIwmQGmevQu31%2Fmpf93noxyUxZ8JsaRQPrgHs8EqBABjk20MixJmQ%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T024731Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGX5ZM32TPU%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=bfef91296fc6f44533d240851b59db9486f50c660c02cfc54cc6c1294554986a (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:852)'),)) Downloading [####################]100.00% 380.0kB/380.0kB (314.7kB/s) raw.png Done... Downloading [####################]100.00% 1.9kB/1.9kB (4.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 1.7kB/1.7kB (8.3kB/s) instrument_instances.png Done... Downloading [####################]100.00% 377.6kB/377.6kB (54.9kB/s) raw.png Done... Downloading [####################]100.00% 91.1MB/91.1MB (601.4kB/s) 10s_video.zip.synapse_download_40887553 Done... Downloading [####################]100.00% 1.7kB/1.7kB (885.1kB/s) instrument_instances.png Done... Downloading [####################]100.00% 373.0kB/373.0kB (323.6kB/s) raw.png Done... Downloading [####################]100.00% 90.8MB/90.8MB (362.0kB/s) 10s_video.zip.synapse_download_42125151 Done... Downloading [####################]100.00% 91.0MB/91.0MB (725.8kB/s) 10s_video.zip.synapse_download_40887549 Done... Downloading [####################]100.00% 1.8kB/1.8kB (3.9MB/s) instrument_instances.png Done... Downloading [####################]100.00% 373.0kB/373.0kB (318.3kB/s) raw.png Done... Downloading [####################]100.00% 1.8kB/1.8kB (5.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 371.4kB/371.4kB (623.1kB/s) raw.png Done... Downloading [####################]100.00% 89.3MB/89.3MB (796.5kB/s) 10s_video.zip.synapse_download_40887556 Done... Downloading [####################]100.00% 3.2kB/3.2kB (6.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 393.0kB/393.0kB (377.5kB/s) raw.png Done... Downloading [####################]100.00% 87.2MB/87.2MB (784.0kB/s) 10s_video.zip.synapse_download_40887518 Done... Downloading [####################]100.00% 89.8MB/89.8MB (590.4kB/s) 10s_video.zip.synapse_download_40887547 Done... Downloading [####################]100.00% 1.8kB/1.8kB (5.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 89.8MB/89.8MB (529.7kB/s) 10s_video.zip.synapse_download_42125137 Done... / Downloaded 3.1GB (2.3MB/s)2020-11-04 10:51:02,398 [retry:52 - DEBUG]: calling
.
at 0x7fd824c3a7b8> resulted in an Exception 2020-11-04 10:51:02,399 [retry:86 - DEBUG]: retrying exception: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2020-11-04 10:51:02,399 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 0.7 seconds... Downloading [####################]100.00% 362.7kB/362.7kB (221.7kB/s) raw.png Done... | Downloaded 3.1GB (2.3MB/s)2020-11-04 10:51:06,262 [retry:52 - DEBUG]: calling
.
at 0x7fd824b23a60> resulted in an Exception 2020-11-04 10:51:06,262 [retry:86 - DEBUG]: retrying exception: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) 2020-11-04 10:51:06,263 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 0.7 seconds... Downloading [####################]100.00% 366.7kB/366.7kB (268.4kB/s) raw.png Done... Downloading [####################]100.00% 363.6kB/363.6kB (661.8kB/s) raw.png Done... Downloading [####################]100.00% 88.7MB/88.7MB (477.8kB/s) 10s_video.zip.synapse_download_40887525 Done... Downloading [####################]100.00% 346.5kB/346.5kB (254.8kB/s) raw.png Done... Downloading [####################]100.00% 85.9MB/85.9MB (783.7kB/s) 10s_video.zip.synapse_download_40887529 Done... Downloading [####################]100.00% 1.6kB/1.6kB (3.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 424.9kB/424.9kB (194.8kB/s) raw.png Done... / Downloaded 3.4GB (2.3MB/s)2020-11-04 10:52:41,601 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 436, in _error_catcher yield File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 518, in read data = self._fp.read(amt) if not fp_closed else b"" File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/http/client.py", line 459, in read n = self.readinto(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/http/client.py", line 503, in readinto n = self.fp.readinto(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 1012, in recv_into return self.read(nbytes, buffer) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 874, in read return self._sslobj.read(len, buffer) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 631, in read v = self._sslobj.read(len, buffer) ConnectionResetError: [Errno 104] Connection reset by peer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 751, in generate for chunk in self.raw.stream(chunk_size, decode_content=True): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 575, in stream data = self.read(amt=amt, decode_content=decode_content) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 540, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/contextlib.py", line 99, in __exit__ self.gen.throw(type, value, traceback) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 454, in _error_catcher raise ProtocolError("Connection broken: %r" % e, e) urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 297, in download_file self._write_chunks(request, completed_futures, transfer_status) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 372, in _write_chunks chunk_data = chunk_response.content File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 829, in content self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b'' File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 754, in generate raise ChunkedEncodingError(e) requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) Downloading [####################]100.00% 86.1MB/86.1MB (718.6kB/s) 10s_video.zip.synapse_download_42125124 Done... Downloading [####################]100.00% 1.2kB/1.2kB (2.7MB/s) instrument_instances.png Done... Downloading [####################]100.00% 85.3MB/85.3MB (612.7kB/s) 10s_video.zip.synapse_download_42125135 Done... Downloading [####################]100.00% 387.2kB/387.2kB (231.6kB/s) raw.png Done... Downloading [####################]100.00% 1.1kB/1.1kB (2.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 85.3MB/85.3MB (645.5kB/s) 10s_video.zip.synapse_download_42125126 Done... Downloading [####################]100.00% 372.0kB/372.0kB (333.3kB/s) raw.png Done... Downloading [####################]100.00% 1.0kB/1.0kB (1.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 357.1kB/357.1kB (258.5kB/s) raw.png Done... Downloading [####################]100.00% 85.8MB/85.8MB (671.7kB/s) 10s_video.zip.synapse_download_40887504 Done... Downloading [####################]100.00% 90.3MB/90.3MB (780.3kB/s) 10s_video.zip.synapse_download_42125142 Done... Downloading [####################]100.00% 2.4kB/2.4kB (4.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 346.4kB/346.4kB (319.0kB/s) raw.png Done... Downloading [####################]100.00% 2.0kB/2.0kB (4.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 320.9kB/320.9kB (128.1kB/s) raw.png Done... Downloading [####################]100.00% 86.0MB/86.0MB (471.8kB/s) 10s_video.zip.synapse_download_40887515 Done... Downloading [####################]100.00% 2.0kB/2.0kB (5.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 366.9kB/366.9kB (310.7kB/s) raw.png Done... Downloading [####################]100.00% 84.6MB/84.6MB (686.3kB/s) 10s_video.zip.synapse_download_40887488 Done... Downloading [####################]100.00% 1.9kB/1.9kB (3.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 368.3kB/368.3kB (248.8kB/s) raw.png Done... Downloading [####################]100.00% 85.5MB/85.5MB (586.4kB/s) 10s_video.zip.synapse_download_40887509 Done... Downloading [####################]100.00% 1.7kB/1.7kB (3.8MB/s) instrument_instances.png Done... Downloading [####################]100.00% 361.9kB/361.9kB (307.5kB/s) raw.png Done... Downloading [####################]100.00% 84.1MB/84.1MB (533.4kB/s) 10s_video.zip.synapse_download_40887493 Done... Downloading [####################]100.00% 1.9kB/1.9kB (7.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 340.8kB/340.8kB (329.7kB/s) raw.png Done... Downloading [####################]100.00% 84.6MB/84.6MB (846.6kB/s) 10s_video.zip.synapse_download_42125136 Done... Downloading [####################]100.00% 2.0kB/2.0kB (3.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 361.3kB/361.3kB (314.0kB/s) raw.png Done... Downloading [####################]100.00% 85.7MB/85.7MB (633.5kB/s) 10s_video.zip.synapse_download_40887492 Done... Downloading [####################]100.00% 86.0MB/86.0MB (595.6kB/s) 10s_video.zip.synapse_download_40887489 Done... Downloading [####################]100.00% 336.2kB/336.2kB (278.5kB/s) raw.png Done... Downloading [####################]100.00% 81.1MB/81.1MB (867.9kB/s) 10s_video.zip.synapse_download_40887459 Done... Downloading [####################]100.00% 1.8kB/1.8kB (5.2MB/s) instrument_instances.png Done... Downloading [####################]100.00% 389.2kB/389.2kB (362.7kB/s) raw.png Done... Downloading [####################]100.00% 82.5MB/82.5MB (694.4kB/s) 10s_video.zip.synapse_download_40887460 Done... Downloading [####################]100.00% 2.2kB/2.2kB (4.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 268.0kB/268.0kB (332.5kB/s) raw.png Done... Downloading [####################]100.00% 82.7MB/82.7MB (475.6kB/s) 10s_video.zip.synapse_download_40887478 Done... Downloading [####################]100.00% 2.3kB/2.3kB (5.3MB/s) instrument_instances.png Done... Downloading [####################]100.00% 353.6kB/353.6kB (650.1kB/s) raw.png Done... | Downloaded 4.4GB (2.4MB/s)2020-11-04 10:59:23,655 [retry:52 - DEBUG]: calling
.
at 0x7fd824c15620> resulted in an Exception ...... ...... omit the middle part ...... ...... / Downloaded 10.9GB (1.4MB/s)2020-11-04 12:38:10,728 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 276, in download_file file_size = _get_file_size(url_info.url) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 203, in _get_file_size return int(res_get.headers['Content-Length']) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/structures.py", line 54, in __getitem__ return self._store[key.lower()][1] KeyError: 'content-length' \ Downloaded 10.9GB (1.4MB/s)2020-11-04 12:43:29,286 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 436, in _error_catcher yield File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 518, in read data = self._fp.read(amt) if not fp_closed else b"" File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/http/client.py", line 459, in read n = self.readinto(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/http/client.py", line 503, in readinto n = self.fp.readinto(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 1012, in recv_into return self.read(nbytes, buffer) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 874, in read return self._sslobj.read(len, buffer) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 631, in read v = self._sslobj.read(len, buffer) ConnectionResetError: [Errno 104] Connection reset by peer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 751, in generate for chunk in self.raw.stream(chunk_size, decode_content=True): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 575, in stream data = self.read(amt=amt, decode_content=decode_content) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 540, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/contextlib.py", line 99, in __exit__ self.gen.throw(type, value, traceback) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 454, in _error_catcher raise ProtocolError("Connection broken: %r" % e, e) urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 297, in download_file self._write_chunks(request, completed_futures, transfer_status) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 372, in _write_chunks chunk_data = chunk_response.content File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 829, in content self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b'' File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 754, in generate raise ChunkedEncodingError(e) requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) Downloading [####################]100.00% 73.5MB/73.5MB (696.5kB/s) 10s_video.zip.synapse_download_42125016 Done... Downloading [####################]100.00% 362.0kB/362.0kB (142.9kB/s) raw.png Done... Downloading [####################]100.00% 73.3MB/73.3MB (785.4kB/s) 10s_video.zip.synapse_download_42125017 Done... Downloading [####################]100.00% 273.8kB/273.8kB (203.5kB/s) raw.png Done... Downloading [####################]100.00% 72.9MB/72.9MB (1.0MB/s) 10s_video.zip.synapse_download_42124985 Done... Downloading [####################]100.00% 1.5kB/1.5kB (18.5kB/s) instrument_instances.png Done... Downloading [####################]100.00% 290.5kB/290.5kB (30.1kB/s) raw.png Done... Downloading [####################]100.00% 73.0MB/73.0MB (221.9kB/s) 10s_video.zip.synapse_download_40887122 Done... Downloading [####################]100.00% 1.4kB/1.4kB (4.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 308.4kB/308.4kB (283.5kB/s) raw.png Done... Downloading [####################]100.00% 71.8MB/71.8MB (304.6kB/s) 10s_video.zip.synapse_download_40887072 Done... Downloading [####################]100.00% 2.0kB/2.0kB (6.4MB/s) instrument_instances.png Done... Downloading [####################]100.00% 298.1kB/298.1kB (183.9kB/s) raw.png Done... Downloading [####################]100.00% 70.6MB/70.6MB (816.9kB/s) 10s_video.zip.synapse_download_42124984 Done... Downloading [####################]100.00% 268.3kB/268.3kB (229.6kB/s) raw.png Done... Downloading [####################]100.00% 69.2MB/69.2MB (867.8kB/s) 10s_video.zip.synapse_download_42124973 Done... \ Downloaded 11.4GB (1.3MB/s)2020-11-04 13:02:35,592 [retry:52 - DEBUG]: calling
.
at 0x7fd824946d90> resulted in an Exception 2020-11-04 13:02:35,592 [retry:86 - DEBUG]: retrying exception: ('Connection aborted.', OSError(0, 'Error')) 2020-11-04 13:02:35,592 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 0.8 seconds... 2020-11-04 13:04:45,959 [retry:52 - DEBUG]: calling
.
at 0x7fd824946d90> resulted in an Exception 2020-11-04 13:04:45,959 [retry:86 - DEBUG]: retrying exception: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/3384422/106b60b8-8d67-4195-a221-a65df6d90b06/raw.png?response-content-disposition=attachment%3B%20filename%3D%22raw.png%22%3B%20filename%2A%3Dutf-8%27%27raw.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHQaCXVzLWVhc3QtMSJGMEQCIFx5F5%2BQysON3i%2FvVTXGoYOqRe2Nkw1CmDIMfaWxtzK1AiANPJ2j%2BUUw%2B5YImZXsetbIxqwCtHzMFnB2iNICUaC5eiq9AwjN%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAAaDDMyNTU2NTU4NTgzOSIM0SSI9tyLsO0h1iJ9KpEDiImrTuEQ4Yl113HPT7xdBCmTQPQ1KvZ%2BkR9JOPskieIeQn4%2BUOkW4oc854mFgc4x06bLsFtnDxnmVOsN5cJU4NSNhGzXaiDn3%2BI52%2Bl6Abniuo1C%2Bln6Wy%2FAItJxY%2Fp4S4VulpUsJYV3zs%2B3vAFhAD7fKP6%2B4CSbjBnlBHKCsx02ho11nH8M%2FGzoOPCeo4zmVjoL9lOPJfTQirY%2BeiWeja7zCU7Gzqu1%2FE0TyP0lFb3zq7djg%2BEJWSJn3hGK%2F090WkPRrvFMbmfonbpaHCIYeH78TZzb9%2FixA9Jz2PZNCLJjkS8OKTxj6aG4SgHHAWEvLJ3MqUlM4m3P12VLX6Vv6eCv4u2D0pFCkmlWV08hHKfYuPHkIgIXX92ZeVjnCw7Ht1I%2F84oOJbCzbYV62GvAOhA%2Fjo%2Bflsp4WHhGqIzdnyqaSsduOEvpRSsadlJxC1BP1BBiLOKbMvcqMoqGrAGzDRPoTnuIyI39EMtisn8Q66FowIAtxtYck1mugpjT8FrHepJrlMzivbieTrIaZdpjzYAw08WI%2FQU67AG0UFc86Bc24Hk7qME6K37au%2F6sx4mPtul0HNr%2Blfo3YgQ0gsaw535fym1Kz%2BW6OQHi3MXuCk5mndv2VIkdEkcdO1gD%2F2l3A2%2Fsn%2FWD0wH58SyDmACSDc5s%2F3%2Ffl5yW97%2FW0zE3grlnBsfxVCap9d2T7BKiE81jdwnE5%2FKvUIFR51ha5OeWSVflCpYDXbamlcUct2IS60qEA3XP6q9X1eNbLJKc3mpXawQhjq%2FCFqCcVYgrWW8Gz6je%2Fp5d5qyfH%2F0XoFxYNt9vckkQy01tHfakO8ySQ7Quvuxw2VtMf4kn4tleuBql%2FHdVNyHzrg%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T050211Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGX3LRL4KOW%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=ea26f52d22c937cdd432ad4988d4a98e0ba7b7abf14a2859104acedb084af808 (Caused by NewConnectionError('
: Failed to establish a new connection: [Errno 110] Connection timed out',)) 2020-11-04 13:04:45,959 [retry:93 - DEBUG]: total wait time 1 seconds ... Retrying in 1.0 seconds... 2020-11-04 13:04:48,298 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1798, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1900, in _download_from_URL exceptions._raise_for_status(response, verbose=self.debug) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/exceptions.py", line 160, in _raise_for_status raise SynapseHTTPError(message, response=response) synapseclient.core.exceptions.SynapseHTTPError: 403 Client Error: Forbidden
AccessDenied
Request has expired
30
2020-11-04T05:02:41Z
2020-11-04T05:04:48Z
73FF1BB769CC7D82
ghuPxETzhr4t/RwjXXPCfAx9S2i0iMNyrqmjTb6r+1O/kd4BaQrfWHXWlPb71429V/WGWqwDYeY=
>>>>>> Request <<<<<< https://s3.amazonaws.com/proddata.sagebase.org/3384422/106b60b8-8d67-4195-a221-a65df6d90b06/raw.png?response-content-disposition=attachment%3B%20filename%3D%22raw.png%22%3B%20filename%2A%3Dutf-8%27%27raw.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHQaCXVzLWVhc3QtMSJGMEQCIFx5F5%2BQysON3i%2FvVTXGoYOqRe2Nkw1CmDIMfaWxtzK1AiANPJ2j%2BUUw%2B5YImZXsetbIxqwCtHzMFnB2iNICUaC5eiq9AwjN%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAAaDDMyNTU2NTU4NTgzOSIM0SSI9tyLsO0h1iJ9KpEDiImrTuEQ4Yl113HPT7xdBCmTQPQ1KvZ%2BkR9JOPskieIeQn4%2BUOkW4oc854mFgc4x06bLsFtnDxnmVOsN5cJU4NSNhGzXaiDn3%2BI52%2Bl6Abniuo1C%2Bln6Wy%2FAItJxY%2Fp4S4VulpUsJYV3zs%2B3vAFhAD7fKP6%2B4CSbjBnlBHKCsx02ho11nH8M%2FGzoOPCeo4zmVjoL9lOPJfTQirY%2BeiWeja7zCU7Gzqu1%2FE0TyP0lFb3zq7djg%2BEJWSJn3hGK%2F090WkPRrvFMbmfonbpaHCIYeH78TZzb9%2FixA9Jz2PZNCLJjkS8OKTxj6aG4SgHHAWEvLJ3MqUlM4m3P12VLX6Vv6eCv4u2D0pFCkmlWV08hHKfYuPHkIgIXX92ZeVjnCw7Ht1I%2F84oOJbCzbYV62GvAOhA%2Fjo%2Bflsp4WHhGqIzdnyqaSsduOEvpRSsadlJxC1BP1BBiLOKbMvcqMoqGrAGzDRPoTnuIyI39EMtisn8Q66FowIAtxtYck1mugpjT8FrHepJrlMzivbieTrIaZdpjzYAw08WI%2FQU67AG0UFc86Bc24Hk7qME6K37au%2F6sx4mPtul0HNr%2Blfo3YgQ0gsaw535fym1Kz%2BW6OQHi3MXuCk5mndv2VIkdEkcdO1gD%2F2l3A2%2Fsn%2FWD0wH58SyDmACSDc5s%2F3%2Ffl5yW97%2FW0zE3grlnBsfxVCap9d2T7BKiE81jdwnE5%2FKvUIFR51ha5OeWSVflCpYDXbamlcUct2IS60qEA3XP6q9X1eNbLJKc3mpXawQhjq%2FCFqCcVYgrWW8Gz6je%2Fp5d5qyfH%2F0XoFxYNt9vckkQy01tHfakO8ySQ7Quvuxw2VtMf4kn4tleuBql%2FHdVNyHzrg%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T050211Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGX3LRL4KOW%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=ea26f52d22c937cdd432ad4988d4a98e0ba7b7abf14a2859104acedb084af808 GET >>> Headers: {'User-Agent': 'synapsecommandlineclient synapseclient/2.2.2 python-requests/2.24.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'userId': 'cz.li@siat.ac.cn', 'signatureTimestamp': '2020-11-04T05:04:47.000Z', 'signature': b'cwtwfBCMheKOmnc7hn4TiWZnb54='} >>> Body: None >>>>>> Response <<<<<<
>>> Headers: {'x-amz-request-id': '73FF1BB769CC7D82', 'x-amz-id-2': 'ghuPxETzhr4t/RwjXXPCfAx9S2i0iMNyrqmjTb6r+1O/kd4BaQrfWHXWlPb71429V/WGWqwDYeY=', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Date': 'Wed, 04 Nov 2020 05:04:47 GMT', 'Server': 'AmazonS3'} >>> Body:
AccessDenied
Request has expired
30
2020-11-04T05:02:41Z
2020-11-04T05:04:48Z
73FF1BB769CC7D82
ghuPxETzhr4t/RwjXXPCfAx9S2i0iMNyrqmjTb6r+1O/kd4BaQrfWHXWlPb71429V/WGWqwDYeY=
Downloading [####################]100.00% 283.9kB/283.9kB (243.7kB/s) raw.png Done... Downloading [####################]100.00% 65.9MB/65.9MB (967.9kB/s) 10s_video.zip.synapse_download_42124969 Done... Downloading [####################]100.00% 313.4kB/313.4kB (51.6kB/s) raw.png Done... Downloading [####################]100.00% 62.8MB/62.8MB (1.0MB/s) 10s_video.zip.synapse_download_42124970 Done... Downloading [####################]100.00% 323.7kB/323.7kB (277.5kB/s) raw.png Done... Downloading [####################]100.00% 58.6MB/58.6MB (801.4kB/s) 10s_video.zip.synapse_download_42124960 Done... Downloading [####################]100.00% 334.7kB/334.7kB (271.2kB/s) raw.png Done... | Downloaded 11.6GB (1.2MB/s)2020-11-04 13:11:22,407 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 276, in download_file file_size = _get_file_size(url_info.url) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 203, in _get_file_size return int(res_get.headers['Content-Length']) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/structures.py", line 54, in __getitem__ return self._store[key.lower()][1] KeyError: 'content-length' Downloading [####################]100.00% 56.2MB/56.2MB (585.4kB/s) 10s_video.zip.synapse_download_42124999 Done... | Downloaded 11.6GB (1.2MB/s)2020-11-04 13:13:24,359 [retry:52 - DEBUG]: calling
.
at 0x7fd840e7c620> resulted in an Exception 2020-11-04 13:13:24,359 [retry:86 - DEBUG]: retrying exception: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) 2020-11-04 13:13:24,359 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 1.5 seconds... Downloading [####################]100.00% 176.4kB/176.4kB (136.9kB/s) raw.png Done... \ Downloaded 11.7GB (1.1MB/s)2020-11-04 13:27:26,690 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 436, in _error_catcher yield File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 518, in read data = self._fp.read(amt) if not fp_closed else b"" File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/http/client.py", line 459, in read n = self.readinto(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/http/client.py", line 503, in readinto n = self.fp.readinto(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 1012, in recv_into return self.read(nbytes, buffer) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 874, in read return self._sslobj.read(len, buffer) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/ssl.py", line 631, in read v = self._sslobj.read(len, buffer) ConnectionResetError: [Errno 104] Connection reset by peer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 751, in generate for chunk in self.raw.stream(chunk_size, decode_content=True): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 575, in stream data = self.read(amt=amt, decode_content=decode_content) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 540, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/contextlib.py", line 99, in __exit__ self.gen.throw(type, value, traceback) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/urllib3/response.py", line 454, in _error_catcher raise ProtocolError("Connection broken: %r" % e, e) urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 297, in download_file self._write_chunks(request, completed_futures, transfer_status) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 372, in _write_chunks chunk_data = chunk_response.content File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 829, in content self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b'' File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/models.py", line 754, in generate raise ChunkedEncodingError(e) requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) Downloading [####################]100.00% 55.9MB/55.9MB (940.9kB/s) 10s_video.zip.synapse_download_42124968 Done... Downloading [####################]100.00% 280.4kB/280.4kB (244.8kB/s) raw.png Done... Downloading [####################]100.00% 55.8MB/55.8MB (885.6kB/s) 10s_video.zip.synapse_download_42124990 Done... Downloading [####################]100.00% 227.4kB/227.4kB (199.6kB/s) raw.png Done... Downloading [####################]100.00% 55.6MB/55.6MB (418.9kB/s) 10s_video.zip.synapse_download_42124952 Done... - Downloaded 11.8GB (1.1MB/s)2020-11-04 13:34:23,623 [retry:52 - DEBUG]: calling
.
at 0x7fd824968d90> resulted in an Exception 2020-11-04 13:34:23,623 [retry:86 - DEBUG]: retrying exception: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /proddata.sagebase.org/3384422/5e475312-fb21-458b-b5df-c6781dd54e7c/raw.png?response-content-disposition=attachment%3B%20filename%3D%22raw.png%22%3B%20filename%2A%3Dutf-8%27%27raw.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHUaCXVzLWVhc3QtMSJHMEUCIQCXLB72tdqiDsQfh7D%2FWnc%2BkDHAc703sR37tWKG251NMAIgQaY7%2B8yw27wkm8NKNOCY0oZVFLxZP32v5Jws0e1QfqEqvQMIzv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwzMjU1NjU1ODU4MzkiDOXzR%2BruzYeVkrgjBCqRA8VctOGV3rOKiFd%2BDHaXLObzREF8JSbGwHMlMFB7sOv1fwC%2Bep9mjvMkCytnEyJuAkcunYyPYz9yqrkZRdo%2B6Sk%2Fh%2B1XbADKTdvrgAVHNW63tywwSMiX%2FDsbTMmdDXAmOhKp18eW5T1pKjsNvh3X7uYK6Vo%2FjcCQOvi1ogh1bFZl7Y8d0WJTE0OJEQjdKGPyEehmpgQiO0d9ACIth2kGYgWWv%2FjRN%2FkSP92fIIy6sK9rOsDYcoVatD4%2FK0uo8SEP2kAcTyM58btwStEUzNkUs2VZ%2F7WpnJeeUNfDX7YGC5OJmTtMKQTRZ8z%2FfGqjysuEODpdFVYFPDyEOLPD4gK7BxcP1GTolr9WfLQ68S7efSMJ2rkj48tiXL40%2Byfm87gfkDx23P4RCsH41Bae8qs%2BjvO6%2FBQweoHp5yZKPTu3qFdeh%2BBp06M0j04jfAISFdHEmcjRE5%2FXMQqFFGEJ66F0riCb1Ae5lGvQjJxbtDuclOS8z2lMOcR6xQlr138EXvxIlpCp1xaB1zJjc1yX%2BYbp5uY%2BMJviiP0FOusBJwbteWK5g%2F%2B%2FNwno%2FOZqnMEogKuuAhRPplwb7RC3hKPOBz1%2Fk44QhVYQ4ESkAeIANxu8Gzc1fd0WOlQ4JbSfhkw1q22Zu77ZPC7u77VwXnP3gosbeECZPviz0c787RHdl0NcBvB%2BrOJxKGS4ovnuY8DPQnYATp3kUaGFSyw5FwUfqSpz2p8TDYTRf6TsN0sYkLZqthvWfcRNMbxqJilV%2BJkoH77V8R6u5f7GQHE8YVOVDhWGr9OQWK3vPqWmaUP%2F0e8OI5ZWqel4u3EHL8mcjJBzFcF4oUYn89Rj6fINow7NeOMZttE20NEMCA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T053213Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGXQPX3JV2O%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1832e2705cf64288141d5a4dda705a08909226957d0a3786580ddff90bf9fbc8 (Caused by NewConnectionError('
: Failed to establish a new connection: [Errno 110] Connection timed out',)) 2020-11-04 13:34:23,623 [retry:93 - DEBUG]: total wait time 0 seconds ... Retrying in 0.8 seconds... 2020-11-04 13:34:25,610 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1798, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1900, in _download_from_URL exceptions._raise_for_status(response, verbose=self.debug) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/exceptions.py", line 160, in _raise_for_status raise SynapseHTTPError(message, response=response) synapseclient.core.exceptions.SynapseHTTPError: 403 Client Error: Forbidden
AccessDenied
Request has expired
30
2020-11-04T05:32:43Z
2020-11-04T05:34:25Z
6E8C1A2E5B15729E
LgVcRo6SXtq0W5BJrwaeevroU7pGgJmRSe2U3N3tHF3a3JUx1RxwgIXaC3fA5j04+2KTf3haAv8=
>>>>>> Request <<<<<< https://s3.amazonaws.com/proddata.sagebase.org/3384422/5e475312-fb21-458b-b5df-c6781dd54e7c/raw.png?response-content-disposition=attachment%3B%20filename%3D%22raw.png%22%3B%20filename%2A%3Dutf-8%27%27raw.png&response-content-type=image%2Fpng&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHUaCXVzLWVhc3QtMSJHMEUCIQCXLB72tdqiDsQfh7D%2FWnc%2BkDHAc703sR37tWKG251NMAIgQaY7%2B8yw27wkm8NKNOCY0oZVFLxZP32v5Jws0e1QfqEqvQMIzv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwzMjU1NjU1ODU4MzkiDOXzR%2BruzYeVkrgjBCqRA8VctOGV3rOKiFd%2BDHaXLObzREF8JSbGwHMlMFB7sOv1fwC%2Bep9mjvMkCytnEyJuAkcunYyPYz9yqrkZRdo%2B6Sk%2Fh%2B1XbADKTdvrgAVHNW63tywwSMiX%2FDsbTMmdDXAmOhKp18eW5T1pKjsNvh3X7uYK6Vo%2FjcCQOvi1ogh1bFZl7Y8d0WJTE0OJEQjdKGPyEehmpgQiO0d9ACIth2kGYgWWv%2FjRN%2FkSP92fIIy6sK9rOsDYcoVatD4%2FK0uo8SEP2kAcTyM58btwStEUzNkUs2VZ%2F7WpnJeeUNfDX7YGC5OJmTtMKQTRZ8z%2FfGqjysuEODpdFVYFPDyEOLPD4gK7BxcP1GTolr9WfLQ68S7efSMJ2rkj48tiXL40%2Byfm87gfkDx23P4RCsH41Bae8qs%2BjvO6%2FBQweoHp5yZKPTu3qFdeh%2BBp06M0j04jfAISFdHEmcjRE5%2FXMQqFFGEJ66F0riCb1Ae5lGvQjJxbtDuclOS8z2lMOcR6xQlr138EXvxIlpCp1xaB1zJjc1yX%2BYbp5uY%2BMJviiP0FOusBJwbteWK5g%2F%2B%2FNwno%2FOZqnMEogKuuAhRPplwb7RC3hKPOBz1%2Fk44QhVYQ4ESkAeIANxu8Gzc1fd0WOlQ4JbSfhkw1q22Zu77ZPC7u77VwXnP3gosbeECZPviz0c787RHdl0NcBvB%2BrOJxKGS4ovnuY8DPQnYATp3kUaGFSyw5FwUfqSpz2p8TDYTRf6TsN0sYkLZqthvWfcRNMbxqJilV%2BJkoH77V8R6u5f7GQHE8YVOVDhWGr9OQWK3vPqWmaUP%2F0e8OI5ZWqel4u3EHL8mcjJBzFcF4oUYn89Rj6fINow7NeOMZttE20NEMCA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201104T053213Z&X-Amz-SignedHeaders=host&X-Amz-Expires=30&X-Amz-Credential=ASIAUXTJYTGXQPX3JV2O%2F20201104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1832e2705cf64288141d5a4dda705a08909226957d0a3786580ddff90bf9fbc8 GET >>> Headers: {'User-Agent': 'synapsecommandlineclient synapseclient/2.2.2 python-requests/2.24.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'userId': 'cz.li@siat.ac.cn', 'signatureTimestamp': '2020-11-04T05:34:24.000Z', 'signature': b'9wbj8hQexLlOmZ3fA0fxMPSjlac='} >>> Body: None >>>>>> Response <<<<<<
>>> Headers: {'x-amz-request-id': '6E8C1A2E5B15729E', 'x-amz-id-2': 'LgVcRo6SXtq0W5BJrwaeevroU7pGgJmRSe2U3N3tHF3a3JUx1RxwgIXaC3fA5j04+2KTf3haAv8=', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Date': 'Wed, 04 Nov 2020 05:34:25 GMT', 'Server': 'AmazonS3'} >>> Body:
AccessDenied
Request has expired
30
2020-11-04T05:32:43Z
2020-11-04T05:34:25Z
6E8C1A2E5B15729E
LgVcRo6SXtq0W5BJrwaeevroU7pGgJmRSe2U3N3tHF3a3JUx1RxwgIXaC3fA5j04+2KTf3haAv8=
Downloading [####################]100.00% 231.8kB/231.8kB (71.2kB/s) raw.png Done... \ Downloaded 11.8GB (1.1MB/s)2020-11-04 13:35:36,914 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 276, in download_file file_size = _get_file_size(url_info.url) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 203, in _get_file_size return int(res_get.headers['Content-Length']) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/structures.py", line 54, in __getitem__ return self._store[key.lower()][1] KeyError: 'content-length' Downloading [####################]100.00% 56.5MB/56.5MB (407.1kB/s) 10s_video.zip.synapse_download_40887085 Done... Downloading [####################]100.00% 2.0kB/2.0kB (6.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 198.7kB/198.7kB (37.3kB/s) raw.png Done... Downloading [####################]100.00% 57.9MB/57.9MB (907.1kB/s) 10s_video.zip.synapse_download_40887065 Done... Downloading [####################]100.00% 1.4kB/1.4kB (3.6MB/s) instrument_instances.png Done... Downloading [####################]100.00% 232.1kB/232.1kB (56.7kB/s) raw.png Done... Downloading [####################]100.00% 58.2MB/58.2MB (708.7kB/s) 10s_video.zip.synapse_download_42124987 Done... Downloading [####################]100.00% 1.8kB/1.8kB (5.0MB/s) instrument_instances.png Done... Downloading [####################]100.00% 255.6kB/255.6kB (249.2kB/s) raw.png Done... / Downloaded 11.9GB (1.1MB/s)2020-11-04 13:41:43,275 [client:1809 - DEBUG]: Retrying download on error: [
] after progressing 0 bytes Traceback (most recent call last): File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1792, in _downloadFileHandle expected_md5=fileHandle.get('contentMd5')) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/client.py", line 1833, in _download_from_url_multi_threaded multithread_download.download_file(self, request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file downloader.download_file(download_request) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 276, in download_file file_size = _get_file_size(url_info.url) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/synapseclient/core/multithread_download/download_threads.py", line 203, in _get_file_size return int(res_get.headers['Content-Length']) File "/home/licaizi/anaconda3/envs/SINet/lib/python3.6/site-packages/requests/structures.py", line 54, in __getitem__ return self._store[key.lower()][1] KeyError: 'content-length' Downloading [####################]100.00% 57.3MB/57.3MB (1.0MB/s) 10s_video.zip.synapse_download_40887069 Done... Downloading [####################]100.00% 1.9kB/1.9kB (5.1MB/s) instrument_instances.png Done... Downloading [####################]100.00% 265.6kB/265.6kB (227.1kB/s) raw.png Done... \ Downloaded 12.0GB (1.0MB/s) ```
@SIAT-HCI The first error during the version check indicates that it was unable to confirm the current latest version of Synapse, it does so to prompt you to upgrade, but the http connection was refused when it attempted to download the version file. Are you running behind a proxy or a firewall that might prevent access to certain URLs? Can you try running the following command from the command line? This will attempt to download the same file outside of synapse and will help to confirm whether something is blocking your access. ``` curl https://raw.githubusercontent.com/Sage-Bionetworks/synapsePythonClient/master/synapseclient/synapsePythonClient ``` Even if the version check was unsuccessful, the synapse client should still attempt the download, but there may be other similar download connections refused but that aren't showing in the default logging state because they are retried. Can you try running the following command and see if any additional error stack traces are printed, and if so paste them here. ``` synapse --debug get -r syn20575265 --downloadLocation "/home/licaizi/DataSet/Robust Medical Instrument Segmentation(2019)" ``` The above command line should do the same as the download.py script you entered above but will enable extra debugging so we can see any errors that may be encountered during login. In particular I wonder if other download connections are being blocked in your network. Thanks, Jordan
@jordank Hi, the disk space has 2.4TB left, I start to download the data by running "python download.py" in the command line, there first shows an Exception that is "Exception in version check: HTTPConnectionPool, ... , Failed to establish a new connection: [Errno 111] connnection refused", then "Welcome, " come out and download begins. However, the download is stuck after about one or two hours later without any log. here is my download script(download.py): ``` import synapseclient import synapseutils FOLDER = "/home/licaizi/DataSet/Robust Medical Instrument Segmentation(2019)" EMAIL = "********" PSW = "********" PRO_ID = "syn20575265" def download_data(email, password, local_folder, pro_id): print("Start downloading") syn = synapseclient.login(email=email, password=password, rememberMe=True) project_id = pro_id all_files = synapseutils.syncFromSynapse(syn, entity=project_id, path=local_folder) print("Finished downloading") if __name__ == '__main__': local_folder = FOLDER email = EMAIL password = PSW proj_id = PRO_ID download_data(email, password, local_folder, proj_id) ```
Hi @SIAT-HCI, Can you show the command and/or Python code you are using to download the data? Also can you confirm that you have sufficient disk space on the disk you are downloading to store the downloaded content. Synapse client version 2.2.2 fixes an error in version 2.2.0 that could cause the client to appear to hang if it ran out of disk space during the download, if you are running 2.2.0 and you have run out of disk space upgrading to the latest version will cause the error to be reported more cleanly. You can upgrade e.g. ``` pip3 install --upgrade synapseclient ``` Thanks.
@AnnikaReinke Hi, I still encounter trouble in downloading data. I am using the synapseclient 2.2. The download process is often stuck without any log. Is there any other download way?
Dear participants, Due to the feedback of some users, synapse was able to fix the download speed issue. If you want to download our data, we strongly recommend you to update your synapse version and use our download script as usual. pip install --upgrade synapseclient Please come back to us if you are still struggling to download our data. With best regards, The ROBUST-MIS team
Hi, downloading has started. I think I was giving wrong project_id. Thanks for your help :)
Hi @aashu, what does it mean "I'm still not able to download the dataset". Could you please provide further details? Did you try to do the following: 1. Did you set the correct id in the download script to "syn20575265"? ```project_id = "syn20575265" # this is the project id of the files.``` 2. Did you enter your correct credentials in the download script? Cheers
Can you please help me out? I am still not able to download the dataset.
@brucehoff All registered synapse users as well as the EndoVis Participants should be able to download the files. Is there anything else we have to change?
@aashu , > But I don't know how to contact the author within synapse platform. You should be able to make contact in this forum or by email, AnnikaReinke@synapse.org. > provide me the alternatives to download the dataset I recommend using the latest Synapse Python client, https://docs.synapse.org/articles/getting_started_clients.html#python
I think it is controlled by @AnnikaReinke. But I don't know how to contact the author within synapse platform. Also it would be great if you could provide me the alternatives to download the dataset since the dataset is very large. thanks.
@aashu Yes, you must contact the person who controls access to the file (or the folder or project the file is in) and ask them to grant you download permission.
Hi, I am getting this error. Can you please tell me how to resolve this? thanks [WARNING] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING: You have READ permission on this file entity but not DOWNLOAD permission. The file has NOT been downloaded. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
@mkalia There are a lot of factors that can affect data transfer speed. A major one is the speed of your internet connection to the stored data. The challenge organizers have stored the data you are downloading in Amazon S3 storage in the US-East region. I found this site that will perform a speed test: https://cloudharmony.com/speedtest-for-aws. I'm curious what Mb/s values you get on the 11th line ("Amazon S3 / us-east-1"). Please be sure to run the test on machine on which you are performing the data download. Can you also perform this test: Using the Synapse Python client, download syn20600600, which is one of the larger files in the data set: ``` >>> import synapseclient >>> syn = synapseclient.Synapse() >>> syn.login() >>> syn.get("syn20600600") ``` This is the result I see, when just downloading to my laptop at home: ``` Downloading [####################]100.00% 149.5MB/149.5MB (2.7MB/s) 10s_video.zip Done... ``` i.e. the data transfer speed is 2.7 MB/s. What is your result?
@brucehoff : Thanks again for replying . Yes I am aware. In the training dataset there are two folders "Proctocolectomy" and "rectal resection". Both the folders have 10 patients each. It has already been approximately 36 hours and I have not completed downloading even two patients (In both Proctocolectomy and rectal resection). This speed will take more than 10 days to download the entire dataset. I have also noticed that the download stops at random times unless I click on the terminal. (Not sure if that's an issue with the display of information on the terminal or the download itself). Will appreciate any help.
@mkalia Are you aware that syn20595092 contains 8439 files, totaling 221 GB and syn20575555 contains 8515 files totaling 253 GB? How long are you finding the download to take?
@brucehoff : Not sure about this stuff. How can one check that ?
What data transfer rate are you experiencing?
@brucehoff The python script is given in the "data" section of this website. I am trying to download testing data with the codes syn20595092 and syn20575555. I am in British Columbia Canada currently.
@mkalia Can you give a little more context? For example, what is the "python script"? What data are you downloading that appears to be slow? What data transfer rate are you experiencing? Can you share any details about the transfer destination, e.g. its geographic location and its network connectivity?
@brucehoff : Thanks for replying. Yes it already is latest version 2.0.0
@mkalia What version of the Synapse Python client are you using? Are you using the latest version? https://pypi.org/project/synapseclient/#history
Hello, The download speed with the provided python script is very slow. Could you provide an alternative way to download data. I will really appreciate that . Thanks :)
A new version of the Synapse Python client was released just yesterday. It has enhancements for downloading data. https://pypi.org/project/synapseclient/ If that doesn't resolve your issue, could you please provide more detail about the problem you are having?
@brucehoff is there any alternate way or link to download this data.? Could you please write me an email with a download alternative? That would be really appreciated . Thank you!
@ihsan149, I would be happy to help. Can you provide the details of the problem you encountered?
Hello, I also encounter problem in downloading data. Can you suggest alternative methods? Thanks,
Drop files to upload
Your web browser must have JavaScript enabled in order for this application to display correctly.
If you are an automated web crawler from a search engine, follow this
AJAX application crawl link
How to download data?
page is loading…