(monitoring) How to check if consumers are working well?

[zhenqianz]

I used Machinery for quite some time now, and I almost have no problem. However one day the consumers stopped consuming. I used Redis as the broker and it was running okay. The problem was fixed after I restarted the service. And it did well for a couple of months.

Then, one day, my service did break again, and the tasks were not consumed again by the consumer. It was also fixed by restarting the service.

The question is, how to check if the consumer is working well? Could I get info on how many tasks are in the queue? (excluding the health-check task and periodic task). Or is there any way I can monitor the consumers?

Is there anyone with the same concern?

NB.
Currently, I made a periodic task to log something, and I check if there were no logs for some time, I would send an alert. I know this is not a good way if you are having too many services, therefore I create this issue to know the proper way to achieve that.

[oz-world]

mark