Celery SQS Consumer Issue: Visibility Timeout and Message Handling #8691
dineshkumar20
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I've implemented a Celery SQS consumer application with a visibility timeout of 3600 seconds configured in the broker_transport_options. The consumer processes messages from an SQS queue produced by another microservice using the send_task method.
I'm facing two main issues:
Visibility Timeout: Despite setting the visibility timeout to 3600 seconds, the SQS message is moving from the "in-flight" state to the "available" state in just over a minute. The processing of the message takes more than 1 minute, and I expected the message to remain in the "in-flight" state during this time.
Exception Handling: When an exception occurs during message processing, the message is removed from the queue instead of being retried. I want the message to be retried in case of an exception.
Below are simplified versions of the relevant code:
producer.py
Consumer:
Processor.py
Could someone guide me on what I might be missing in the configuration to address these issues?
Beta Was this translation helpful? Give feedback.
All reactions