Skip to content
This repository has been archived by the owner on Feb 15, 2024. It is now read-only.

Verifiers are getting hard to sync up #42

Open
krypdkat opened this issue Nov 22, 2021 · 1 comment
Open

Verifiers are getting hard to sync up #42

krypdkat opened this issue Nov 22, 2021 · 1 comment

Comments

@krypdkat
Copy link

Recently, I saw many verifiers got dropped because they lost sync and never get back to a normal state again (despite the owners trying to reboot the machine, spin up new VPS, pray...). After digging for a while, I found out that the reason those verifiers kept losing synced state is because of lack of block vote, which we can increase the speed of fetching here:
https://github.com/n-y-z-o/nyzoVerifier/blob/master/src/main/java/co/nyzo/verifier/Verifier.java#L526-L528
Those time numbers are not suitable for the current mesh size anymore.

This would be a trade-off between performance and stability but I guess we have to choose stability. Currently, I'm using these numbers and it works fine (1/4 of original numbers). CPU usage got spike a little bit after starting nyzo service (under 20 sec).

frozenEdge.getVerificationTimestamp() < System.currentTimeMillis() - 7500L &&
                            lastVoteRequestTimestamp < System.currentTimeMillis() - 1000L

tunable params maybe nice to have.

@EggPool
Copy link

EggPool commented Nov 22, 2021

Unsure of the best params to use, but agree with a need for tunable params.

Added your suggests plus a third one to current CE version, 611002
https://github.com/Open-Nyzo/nyzoVerifier

New params, with defaults:

catchup_trigger_ms=30000
Milliseconds. After how large a drift from frozen edge should the verifier ask for missing votes. 30000 = 30 sec. 
I wouldn't go too close to a block time. I'd say 15 sec min to avoid too much stress.

catchup_request_delay_ms=4000
Milliseconds as well. After one request for missing votes, how long to wait for a new one. 4s by default then.

catchup_request_count=10
When requesting missing votes, how many random verifiers to ask to? Beware: the more you raise, the more that takes time, fills your queue with messages to process, and risks you being blacklisted on other verifiers.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants