Hello,
I've been trying to download multiple files through the API (python and command line options). The following command worked great, but downloads appeared to stall after completing ~99% of the files:
```
synapse -u username -p token get -r --multiThreaded --downloadLocation output_dir syn26273710
```
For the remaining files (~ 10 files), I've been trying to call `synapse get` multiple times in parallel through bash/python scripts with different, single synapse IDs for each command. I recently started getting the following error. Any suggestions how to avoid the error and lockout. Is there a better way to download remaining files via API?
Thanks,
Chris
```
synapse -u username -p token get --multiThreaded --downloadLocation output_dir syn31447718
synapse -u username -p token get --multiThreaded --downloadLocation output_dir syn31114805
synapse -u username -p token get --multiThreaded --downloadLocation output_dir ...
synapse -u username -p token get --multiThreaded --downloadLocation output_dir ...
additional synapse get SynID1 calls ...
synapse get SynID2...
synapse get SynID3...
...
Traceback (most recent call last):
File "/opt/conda/envs/snakemake/bin/synapse", line 10, in
sys.exit(main())
^^^^^^
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/__main__.py", line 1212, in main
perform_main(args, syn)
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/__main__.py", line 1114, in perform_main
login_with_prompt(syn, args.synapseUser, args.synapsePassword, silent=True)
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/__main__.py", line 1131, in login_with_prompt
_authenticate_login(syn, user, password, silent=silent, rememberMe=rememberMe, forced=forced)
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/__main__.py", line 1191, in _authenticate_login
syn.login(user, **login_kwargs_with_secret)
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/client.py", line 414, in login
self.credentials = credential_provider_chain.get_credentials(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/core/credentials/credential_provider.py", line 223, in get_credentials
creds = provider.get_synapse_credentials(syn, user_login_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/core/credentials/credential_provider.py", line 41, in get_synapse_credentials
return self._create_synapse_credential(syn, *self._get_auth_info(syn, user_login_args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/core/credentials/credential_provider.py", line 46, in _create_synapse_credential
retrieved_session_token = syn._getSessionToken(email=username, password=password)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/client.py", line 485, in _getSessionToken
session = self.restPOST('/session', body=json.dumps(req), endpoint=self.authEndpoint,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/client.py", line 4047, in restPOST
response = self._rest_call('post', uri, body, endpoint, headers, retryPolicy, requests_session, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/client.py", line 4014, in _rest_call
self._handle_synapse_http_error(response)
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/client.py", line 3984, in _handle_synapse_http_error
exceptions._raise_for_status(response, verbose=self.debug)
File "/opt/conda/envs/snakemake/lib/python3.11/site-packages/synapseclient/core/exceptions.py", line 160, in _raise_for_status
raise SynapseHTTPError(message, response=response)
synapseclient.core.exceptions.SynapseHTTPError: 423 Client Error:
You are locked out from making any additional login attempts for 338665 milliseconds
```
Created by Chris Rhodes crhodes4 Thanks Jared! This worked great and resolved the issue.
I have a quick related question: When playing around with credentials/credential files, I cached credentials with either ```>>> synapseclient.login(rememberMe=True)``` or ```$synapse login --rememberMe```. Both behaved as expected, but I'm seeing the warning:
```
[WARNING] The rememberMe parameter will be deprecated by early 2024. Please use the ~/.synapseConfig or SYNAPSE_AUTH_TOKEN environmental variable to set up your Synapse connection.
```
Just to confirm, when the new changes take effect in 2024, the multiple login attempts (root cause of 423 Client Error above) will still be allowed for registered/approved users that have properly set up ```~/.synapseConfig``` or ```SYNAPSE_AUTH_TOKEN environmental variable```, correct? I'd be surprised if this capability was removed, but wanted to bring this up proactively in case I'm missing something.
Thanks again,
Chris Hi Chris,
You are likely best served by creating a credentials file for Synapse:
https://r-docs.synapse.org/articles/manageSynapseCredentials.html
By repeatedly logging in with your username and password, you are hitting the limit on login attempts. This limit is to prevent spammers.
By creating a credentials file, you should bypass this.
Regards,
Jared