Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve lily task retry logic #1163

Open
1 task
frrist opened this issue Mar 15, 2023 · 2 comments
Open
1 task

Improve lily task retry logic #1163

frrist opened this issue Mar 15, 2023 · 2 comments
Assignees
@Terryhung @birdychang
Labels
P1 P1: Must be resolved

Comments

@frrist
Copy link
Member

frrist commented Mar 15, 2023

Description

When running lily as a distributed indexer and a job fails to complete all tasks successfully, lily reschedules all tasks to be retried. This is inefficient in some cases since some of tasks may have been completed successfully. Lily should instead reschedule only the failed tasks to be retried.

Acceptance criteria

  • Only reschedule failed tasks to rety

Where to begin
@frrist frrist added the P1 P1: Must be resolved label Mar 15, 2023
@frrist frrist mentioned this issue Mar 16, 2023
Merged
@kasteph
Copy link
Contributor

kasteph commented Mar 16, 2023

closed from #1162

@kasteph kasteph closed this as completed Mar 16, 2023
@birdychang
Copy link
Contributor

#1162 will stop all task retries when there's no fatal error. We still need a better solution to retry only fail tasks

@birdychang birdychang reopened this Mar 16, 2023
@frrist frrist removed their assignment Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Terryhung Terryhung

@birdychang birdychang

Labels
P1 P1: Must be resolved
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@kasteph @Terryhung @frrist @birdychang