Hi,
May I ask how can I download the TESLA datasets after I have been granted access to them please? Is there a tutorial about how to link synapse account to my personal AWS account? For downloading the TESLA datasets, I am using the code in https://www.synapse.org/#!Synapse:syn21048999/wiki/607070 to my local server. However, I got the following error:
@biodebian:/media/projects/tesla_data$ python test.py
Welcome, XI REN!
Traceback (most recent call last):
File "test.py", line 9, in
syn8271842 = syn.get(entity='syn8271842')
File "/home/renxi/anaconda3/lib/python3.8/site-packages/synapseclient/client.py", line 717, in get
return self._getWithEntityBundle(entityBundle=bundle, entity=entity, **kwargs)
File "/home/renxi/anaconda3/lib/python3.8/site-packages/synapseclient/client.py", line 833, in _getWithEntityBundle
self._download_file_entity(downloadLocation, entity, ifcollision, submission)
File "/home/renxi/anaconda3/lib/python3.8/site-packages/synapseclient/client.py", line 895, in _download_file_entity
downloadPath = self._downloadFileHandle(entity.dataFileHandleId, objectId, objectType, downloadPath)
File "/home/renxi/anaconda3/lib/python3.8/site-packages/synapseclient/client.py", line 1845, in _downloadFileHandle
downloaded_path = self._download_from_url_multi_threaded(fileHandleId,
File "/home/renxi/anaconda3/lib/python3.8/site-packages/synapseclient/client.py", line 1890, in _download_from_url_multi_threaded
multithread_download.download_file(self, request)
File "/home/renxi/anaconda3/lib/python3.8/site-packages/synapseclient/core/multithread_download/download_threads.py", line 232, in download_file
downloader.download_file(download_request)
File "/home/renxi/anaconda3/lib/python3.8/site-packages/synapseclient/core/multithread_download/download_threads.py", line 276, in download_file
file_size = _get_file_size(url_info.url)
File "/home/renxi/anaconda3/lib/python3.8/site-packages/synapseclient/core/multithread_download/download_threads.py", line 203, in _get_file_size
return int(res_get.headers['Content-Length'])
File "/home/renxi/anaconda3/lib/python3.8/site-packages/requests/structures.py", line 54, in __getitem__
return self._store[key.lower()][1]
KeyError: 'content-length'
Any advice would be highly appreciated. Thank you very much.
Xi
Created by XI REN renxinus Hi all,
I have created AWS account and set up EC2 instance.
I also set "credentials" file at C:\Users\USERNAME\.aws\
[default]
aws_access_key_id = XXXXXXXXXXXX
aws_secret_access_key = XXXXXXXXXX
region = us-east-1
However, when I run the python script from wiki
syn = synapseclient.login("twpcsie", "XXXXX") ### please enter your Synapse username and password.
FOLDER = "syn23446508" ### this is the 'TESLA phase 1 data release' folder under which all the data is hosted
sts_credentials = syn.restGET(f"/entity/{FOLDER}/sts?permission=read_only") ### this is your 12hr temporary credentials to access the data based on your synapse ID
client = boto3.client(
's3',
aws_access_key_id=sts_credentials['accessKeyId'],
aws_secret_access_key=sts_credentials['secretAccessKey'],
aws_session_token=sts_credentials['sessionToken'],
)
query = syn.tableQuery( " SELECT * FROM syn23534327 WHERE ( ( modifiedBy = 3382314 ) AND ( parentId = 'syn10141118' ) )" ) ### get list of lung cancer file IDs
query_df = query.asDataFrame()
query_IDs = query_df['id'].values
for syn_id in query_IDs:
ent = syn.get(syn_id, downloadFile=False)
client.download_file(ent._file_handle['bucketName'],
ent._file_handle['key'],
ent.name) ### the file will be downloaded to your current folder
It turns out :
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Any possible miss I haven't noticed? Is there any tutorial to link synapse account to personal AWS account Hi all,
To download TESLA data, you need to create an AWS account.
See the link below for more details :)
https://www.synapse.org/#!Synapse:syn21048999/wiki/607070
Thank you,
Jeong-Han Seo Has this been resolved?
I also got the following error in both command line and R.
Error in value[[3L]](cond) : 'content-length'
Thanks,
Jeong-Han Seo hello all,
Sorry for the delays. Thanks for filing this issue - I am going to look into this.
Tom I have the same issue running python client. I even wasn't able to direct download the file to my local PC using by clicking the download link. it shows the following information in the browser:
I have the same issue! Did you get it resolved? I have the same issue! Did you get it resolved? I've had a similar issue recently. Were you able to sort this out?
thanks,
Charles
Drop files to upload
Problem in downloading TESLA datasets page is loading…