Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEATURE: Streaming endpoint for xref matches #3520

Open
tillprochaska opened this issue Dec 5, 2023 · 0 comments
Open

FEATURE: Streaming endpoint for xref matches #3520

tillprochaska opened this issue Dec 5, 2023 · 0 comments
Labels
backend Issues related to Aleph’s backend, API, CLI etc. feature-request Requests for new features or enhancements of existing features

Comments

@tillprochaska
Copy link
Contributor

tillprochaska commented Dec 5, 2023

Is your feature request related to a problem? Please describe.
There are cases when it is necessary to process xref matches offline, e.g. in order to use a custom scoring method. Currently, there’s no easy and scalable way to fetch all xref results.

Describe the solution you'd like
There should be a streaming API endpoint similar that allows to stream xref matches similar to the entity streaming endpoint.

Describe alternatives you've considered

  • Aleph has an option to export xref matches. This doesn’t work well for large numbers of matches (100k).

  • The default xref API is limited to a maximum result set of 10k matches, even when using pagination.

Additional context
Based on user feedback

@tillprochaska tillprochaska added backend Issues related to Aleph’s backend, API, CLI etc. feature-request Requests for new features or enhancements of existing features labels Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Issues related to Aleph’s backend, API, CLI etc. feature-request Requests for new features or enhancements of existing features
Projects
None yet
Development

No branches or pull requests

1 participant