Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TTAHUB-1896] Rework worker and resource processer for increased stability and complete processing #1640

Open
wants to merge 22 commits into
base: main
Choose a base branch
from

Conversation

GarrettEHill
Copy link
Collaborator

Description of change

Add awaits to the processing of queues

How to test

Issue(s)

Checklists

Every PR

  • Meets issue criteria
  • JIRA ticket status updated
  • Code is meaningfully tested
  • Meets accessibility standards (WCAG 2.1 Levels A, AA)
  • API Documentation updated
  • Boundary diagram updated
  • Logical Data Model updated
  • Architectural Decision Records written for major infrastructure decisions
  • UI review complete

Production Deploy

  • Staging smoke test completed

After merge/deploy

  • Update JIRA ticket status

@GarrettEHill GarrettEHill changed the title [TTAHUB-1896] add awaits to the processing of the queues [TTAHUB-1896] Rework worker and resource processer for increased stability and complete processing Aug 1, 2023
Copy link
Collaborator

@AdamAdHocTeam AdamAdHocTeam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran for a year seeing alot of titles being updated along with the new mime type column.

Some tests are still failing.

const onCompletedAWSElasticsearchQueue = (job, result) => {
const onFailedAWSElasticsearchQueue = async (job, error) => {
auditLogger.error(`job ${job.data.key} failed with error ${error}`);
await job.retry();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we should be manually triggering retries. This should be handled automatically by the bull framework when an exception is thrown.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this in a few places below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants