Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove streaming downloader #383

Open
orangejulius opened this issue Feb 17, 2020 · 2 comments
Open

Remove streaming downloader #383

orangejulius opened this issue Feb 17, 2020 · 2 comments

Comments

@orangejulius
Copy link
Member

In addition to a standard download, this importer supports streaming data directly from a ZIP archive over HTTP.

When we originally wrote it, we thought the idea of every part of Pelias supporting a "streaming" mode would lead to great functionality. In practice, it just made things brittle, and we generally prefer a discrete download step followed by an import process that does not use the network (except possibly to contact Elasticsearch).

Removing this importer's streaming mode would simplify the code and remove lots of issues we've had with streaming ZIP files, which are inherently hard to stream.

@orangejulius orangejulius transferred this issue from pelias/docker Feb 25, 2020
@orangejulius
Copy link
Member Author

orangejulius commented Feb 25, 2020

Somehow I originally made this issue in the pelias/docker repository. It was supposed to go here, in the Geonames importer repository. 🤷‍♂️

@missinglink
Copy link
Member

Ooh transferring issues between repos... fancy!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants