Cache PUB internal calls

[iszulcdeepsense]

On each Job call, PUB component makes a request to Lifecycle to check if a user is allowed to speak to a job and to get the internal address of the job to forward the request to. Lifecycle, in turn, makes calls to a database, which can perform bad under higher load.

To improve PUB's performance, these internal calls could be cached with a short time-to-live (eg. 1 minute).
The composite key of this cache composes of: user auth token, job name, job version, job endpoint.

It can be cached either by PUB or by Lifecycle, though it makes more sense to cache it by PUB.

[JosefAssadERST]

which can perform bad under higher load

Has this actually been the case?

I'm not against caching at all, but I typically prefer to try as many other things as possible before. Off the top of my head:

• Consolidating DB calls (e.g. if we always make the same two calls in succession, group them in one)
• Checking if the connection pooler is doing the right thing keeping the connection "warm" so we reduce connection overhead
• Measuring how much of the performance budget on a DB connection is in fact unavoidable infrastructure latency (VPN, network latency, other encryption such as TLS)

When we start caching, we will pay a price in more difficult troubleshooting later, and easier to make mistakes which can cause bugs or security issues (e.g. anything touching user accounts will need to remember to update the cache also).

[iszulcdeepsense]

Has this actually been the case?

To be honest, it's just my conjecture. It's been proven that a single request to Racetrack v2 can be slower than v1 due to the external database.
This idea is just a trick to mitigate this, but I don't think it's a problem right now.

At this point, let's leave it as it is. This issue can be even classified as a "premature optimization". Plus, I agree, this could make troubleshooting and testing more difficult.