Headless authservice endpoints not getting updated

[shrutilamba]

I am facing issues while scaling my authservice. My authservice pods frequently scale up an down, and intermittently I see issues where in the membership total isnt getting updated at ambassador. Example, my auth service pods scaled down from 21 to 20 pods, however, ambassador still taking the old value of 21, which is leading to 5xx at my service as one pod which is now terminated is still being considered by ambassador.
Seems like some lag in service discovery.
Does anyone has an idea what could be the issue here?
I am using endpoint resolver and using a headless service for authservice.
Using version v3.7

However, if I move to service resolver, this works correctly and endpoints are getting correctly updated. Any workaround to make it work with endpoint resolver?