Follower getting 503 Error

[MahmudurKhan]

My Environment

• ArangoDB Version: 3.9.3
• Deployment Mode: Single Server | Follower

i have a follower instance of arangodb running and i got this error
stopping applier: got invalid response from leader at tcp://<example ip>:8529 for URL /_api/replication/logger-state?serverId=143424415998203: HTTP 503: Service Unavailable - {"error":true,"errorNum":503,"errorMessage":"service unavailable due to startup or maintenance mode","code":503}

after this error the follower did not retry to connect any more. However the follower we verified that the follower does indeed retries when the leader is offline (for any reason).

I want to know what causes this error and how can we prevent it from happening.

Thank you

[dothebart]

Hi,
please note that 3.9 is EOL for quiet a while now.
I can only advise to at least update to 3.9.12 to get any fixes made in the 3.9 release cycle and better go for an upgrade to 3.11 (after inspecting whether the changelog has things that concern you)

Most probably the leader has rotated away logs required for the follower to download, hence a full resync is required here.
All of these waranties can only be given in full clusters, were replication is ensured to take place. So increasing the bandwith, memory, and disk i/o bandwith of the follower may improve the situation.