Skip to content
This repository has been archived by the owner on Oct 23, 2020. It is now read-only.

Aggregation worker dies; only fixable by Qu restart #231

Open
marcesher opened this issue Nov 10, 2014 · 1 comment
Open

Aggregation worker dies; only fixable by Qu restart #231

marcesher opened this issue Nov 10, 2014 · 1 comment

Comments

@marcesher
Copy link
Contributor

  1. Start an aggregation
  2. Kill mongos
  3. Restart mongos

Actual behavior: The worker dies, because its error handler itself throws an error when trying to reset the job because it can't communicate with mongo

When this happens, that Qu instance's aggregation worker is effectively dead and will not process additional aggregations. This results in a buildup of aggregations until that Qu instance is restarted

Expected behavior:

  1. The error handler should not throw an error, or, if it does, the worker should be restarted
  2. The worker's currently processing job, at the time of failure, would get restarted when a) the worker is restarted and b) able to communicate with mongo successfully
@marcesher
Copy link
Contributor Author

@m3brown found this, which may be useful in supervising the worker: https://github.com/MichaelDrogalis/dire

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant