Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to scrap from /metrics when there are multiple instances behind a load balancer #439

Open
melenaos opened this issue Sep 18, 2023 · 1 comment

Comments

@melenaos
Copy link

I have installed the nuget and I get metrics at https://example.com/metrics, but since I have multiple web server instances, in every Prometheus request the load balancer respond the metrics from different instance every time.

That way my counters have a portion of the counter from one instance and a portion of the other instance.
They should be either

  1. two different scrap jobs, or
  2. combined together

To do the first I need to create two jobs in Prometheus config that access the two instances independently, and then join them in query.
Unfortunately, this is not possible in Azure Web Apps because to access a specific instance the request must set a cookie, and prometheus doesn't support sending the cookie 'ARRAffinity=....' (only auth cookies).

I wish if there was a way to do the second. Maybe with using a Redis type of cache. That way if two processes are setup to use the same Redis server and the same Key, they would increase the same counter and their /metrics would return the same data.

@imclint21
Copy link

Good point, does its posible to use Azure Cache for Redis by example?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants