Hi,
I was granted access to the TESLA data phase 1 release around 4 hours ago.
I can see the data access is a "green" symbol on the browser.
However, when I try to use a python script provided by https://www.synapse.org/#!Synapse:syn21048999/wiki/607070, I get an error message "An error occurred (403) when calling the HeadObject operation: Forbidden".
How can I download these data?
Thanks ahead,
Martin Liu
Created by ChiaHsin Liu MartinLiuTaiwan @xengie.doan TESLA_19_1.fastq.gz and patient09_tumor_rna_1.fq is available for me now. I've download them , MD5 check is OK. Thanks again. @xengie.doan I retried the download for the other synapse IDs that I mentioned apart from TESLA_19_1.fastq.gz (syn8273343) and patient09_tumor_rna_1.fq (syn17090961) and they worked this time. I was also able to download TESLA_19_1.fastq.gz and patient09_tumor_rna_1.fq. Must have been an error in my script. Thank you for your help! @steven.dea What issues did you have with files other than the ones fixed above (TESLA_19_1.fastq.gz (syn8273343) and patient09_tumor_rna_1.fq (syn17090961) ? I just tried to download "syn17091013" and "syn17091005" and it worked for me. Please share the script and your environment so I can try to recreate it. @lk_bioinfo @MartinLiuTaiwan I have fixed the TESLA_19_1.fastq.gz and patient09_tumor_rna_1.fq files and have tested download via STS token to my test EC2. Please let me know if you run into any other issues @xengie.doan I modified python code print bucketName of all files,TESLA_19_1.fastq.gz and patient09_tumor_rna_1.fq are in proddata.sagebase.org ,not like the others. I get a 403 error when I download them. Please tag me when you fix it.
Thanks a lot for your help. @steven.dea We are still working on a fix and will update you when it is completed! Hello,
I am also having issues downloading the following syn IDs: "syn8273343", "syn16804036", "syn17090961", "syn17091013", "syn17091005", "syn17091007". These appear to coincide with what has been previously mentioned in this thread (TESLA_19_1.fastq.gz and pairs of RNA files). Is there a documented fix for this?
Thank you! @GLeoni It is not possible without an AWS account, hopefully the documentation for creating one is helpful. @xengie.doan
Thank you for your tip. I don't have an AWS account is it possible to download the data without it? @MartinLiuTaiwan thank you for sharing the script, it looks like somehow TESLA_19_1.fastq.gz did not get processed the same as the other files and I am looking into how to fix it now!
@OlaNilsson what command did you run to look for Patient 10? The synapse ID's are syn8272295 and syn8272307 and they came up as part of the melanoma 1 query that was shared above.
@GLeoni Currently R is not supported, you would need to adapt these lines from Python into R:
```
import boto3
sts_credentials = syn.restGET(f"/entity/{FOLDER}/sts?permission=read_only") ### this is your 12hr temporary credentials to access the data based on your synapse ID
client = boto3.client(
's3',
aws_access_key_id=sts_credentials['accessKeyId'],
aws_secret_access_key=sts_credentials['secretAccessKey'],
aws_session_token=sts_credentials['sessionToken'],
)
client.download_file(ent._file_handle['bucketName'],
ent._file_handle['key'],
ent.name)
```
Dear all
I'm trying to use the R script without success.
the command is
library(synapser)
library(synapserutils)
synLogin('GLeoni', 'my password')
files <- synapserutils::syncFromSynapse("syn23446508")
and the error is
Error in value[[3L]](cond) : File download failed during sync
do you have any suggestion ?
thanks
Guido @xengie.doan
Also, I can't seem to find the data for patient 10 ?
@xengie.doan the downloadFile=True works, but then you have to move it from the cache dir to the destination folder. By the way I also had some problems with ["syn17090961","syn17091013","syn17091005","syn17091007", I used the same trick to download those.
Hi @xengie.doan ,
I had requested TESLA_19_1.fastq.gz.
Its ID is syn8273343 under a parent ID syn8262420 according to https://www.synapse.org/#!Synapse:syn8262420.
However, when I execute the following code:
query = syn.tableQuery( " SELECT * FROM syn23534327 WHERE ( ( modifiedBy = 3382314 ) AND ( parentId = 'syn16810855' OR parentId = 'syn8262420' ) ) " ) ### get list of 'melanoma 1' file IDs
query_df = query.asDataFrame()
query_IDs = query_df['id'].values
>>> query_IDs
array(['syn8271842', 'syn8271886', 'syn8271888', 'syn8271903',
'syn8271922', 'syn8271939', 'syn8272274', 'syn8272284',
'syn8272295', 'syn8272307', 'syn8272316', 'syn8272341',
'syn8273290', 'syn8273306', 'syn8273311', 'syn8273324',
'syn8273356', 'syn8313637'], dtype=object)
There is no syn8273343 in the query_IDs.
Does this mean TESLA_19_1.fastq.gz is not under this parent ID ?
@MartinLiuTaiwan What files/folders do you requested data access for? When I check where the files on Synapse, I can see the file within the folder so I would need some more information to find the issue.
@OlaNilsson could you share the error or problem more specifically? It is true that if you use the `downloadFile=True` flag from syn_get() usually means that the file will be downloaded into your Synapse cache, but I have not tested it with this data and instead download with this step: `client.download_file(ent._file_handle['bucketName'], ent._file_handle['key'], ent.name) `. If you use the client.download_file() are you able to download? also had problems with some RNA files, ran the following, them moved the downloaded files from the cacheDir
query_IDs = ["syn17090961","syn17091013","syn17091005","syn17091007"]
for syn_id in query_IDs:
ent = syn.get(syn_id, downloadFile=True) Had the same problem, I'm downloaded like this:
ent=syn.get("syn8273343", downloadFile=True)
ent.cacheDir has the file direction , just move it to your home Hi,
I meet a problem with downloading TESLA_19_1.fastq.gz.
Its id is syn8273343 under the parent folder syn8262420 (according to the table).
When I execute the download instruction code, I found there is no syn8273343 under syn8262420.
Thanks ahead,
Martin Liu
Hi @xengie.doan, thanks for helping, I will try to recreate a EC2.
Martin Hi @MartinLiuTaiwan , what region did you create the EC2 in? It should in `us-east-1`, the same region the TESLA data is hosted in. I apologize if the instructions were not clear, and I have recently updated them.