New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AMQP for working queues between SCAN, INDEX, NLP #1374

Open

bamthomas opened this issue Apr 9, 2024 · 1 comment

Labels

stale

Collaborator

bamthomas commented Apr 9, 2024

Is your feature request related to a problem? Please describe.

We have task queue that is either:

Memory
Redis
AMQP

The parameter is batchQueueType

Then we have queueType that are used to instantiate queues used between pipeline tasks (scan, index, deduplicate...).

Describe the solution you'd like

We could imagine:

to have a single queueType parameter that is used by both
to consider each extract of a file/document as a task and use task queues, avoiding these two mecanisms

The text was updated successfully, but these errors were encountered:

bamthomas mentioned this issue

refactor: migrate batch search to AMQP #1080

Closed

11 tasks

github-actions bot commented May 20, 2024

This issue is stale because it has been open for 40 days with no activity.

github-actions bot added the stale label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment