Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make robust against network problems #16

Open
iimog opened this issue Jan 16, 2019 · 0 comments
Open

Make robust against network problems #16

iimog opened this issue Jan 16, 2019 · 0 comments
Projects

Comments

@iimog
Copy link
Member

iimog commented Jan 16, 2019

While I was trying to reproduce #15 I repeatedly got the following error:

WebEnv value not found in post output
Db value not found in fetch input
[01-16 01:09:46] [ReferenceDbCreator] 
[01-16 01:09:46] [ReferenceDbCreator] ERROR: Downloading fasta sequences for batch: 3901 - 4000 failed

This occured at different batches so I assume it is a network related issue. However, a single network error should not stop the whole pipeline from working. Instead it would be desirable to have some mechanism that retries failed downloads after a short period of time (for a limited number of times). This would make the whole process much more robust against common network and connectivity issues (including NCBI rate limits).

@iimog iimog added this to To do in 1.0.0 Mar 15, 2019
iimog added a commit that referenced this issue Nov 2, 2021
Retry to download failed batches, Issues #16 & #31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
1.0.0
  
To do
Development

No branches or pull requests

1 participant