Support Zeebe StreamActivatedJobs API

[jwulf]

SDK Component

Zeebe

The Zeebe Gateway has a new RPC StreamJobs to open a long-lived stream that essentially "pushes" jobs to the worker.

This obviates polling loops.

camunda/zeebe#14152

Environment variable to enable it:

ZEEBE_CLIENT_WORKER_STREAMENABLED

See also the documentation about the general concept, including how back pressure works and can be implemented for your custom client: https://stage.docs.camunda.io/docs/next/components/concepts/job-workers/#job-streaming

[aleksander-dytko]

Hi @jwulf, do I assume correctly that this is targeted for 8.5 release?

[jwulf]

Hi @aleksander-dytko, it will probably soon come after the 8.5 release, asynchronously. There is a lot of work across the entire SDK for the 8.5 release.

[aleksander-dytko]

Thanks @jwulf. Do you know the rough timeline for this?
The customer raised a Feature Request here and I wanted to provide them with an update

[aleksander-dytko]

@jwulf @akeller do we know when this item will be delivered? I wanted to update the customer in the feature request

[jwulf]

This feature is under development, I estimate that it will be available at the end of May. 

In the meantime, would you be able to share the current worker creation parameters? There may be some tuning that can be done with the existing long polling that could mitigate the resource exhaustion. 

Some ideas that occur to me are to extend the long poll period, and also to add a randomised offset to it so that different workers that are started at the same time do not all poll at once.