satellite/overlay decrease NodeCheckInWaitPeriod

[littleskunk]

What: Decrease NodeCheckInWaitPeriod from 2h down to 1h10m

Why: NodeCheckInWaitPeriod is designed to suppress unnecessary DB commits. At the same time, we want to allow the storage node to update its last_contact_success once every 2 hours. It turns out as long as the value is set to 2 hours the storage node has a high chance to get last_contact_success updates once every 3 hours because the second check-in is almost perfect on the 2 hour mark. This is a bit unfair. It means with just 1 hour of downtime the storage node might end up with an last_contact_success that is more than 4 hours old. At that point repair kicks in and moves pieces away from the node.

Reducing NodeCheckInWaitPeriod to 1h10m will make sure the node is allowed to checkin every 2 hours and even accounts for a possible restart after 1h30m or so. It would still commit the node checkin.

Please describe the tests:

• Test 1:
• Test 2:

Please describe the performance impact:

Code Review Checklist (to be filled out by reviewer)

• NEW: Are there any Satellite database migrations? Are they forwards and backwards compatible?
• Does the PR describe what changes are being made?
• Does the PR describe why the changes are being made?
• Does the code follow our style guide?
• Does the code follow our testing guide?
• Is the PR appropriately sized? (If it could be broken into smaller PRs it should be)
• Does the new code have enough tests? (every PR should have tests or justification otherwise. Bug-fix PRs especially)
• Does the new code have enough documentation that answers "how do I use it?" and "what does it do?"? (both source documentation and higher level, diagrams?)
• Does any documentation need updating?
• Do the database access patterns make sense?

[onionjake]

Does it already have jitter as well? Should probably have some random jitter to prevent thundering herds?

[littleskunk]

Does it already have jitter as well? Should probably have some random jitter to prevent thundering herds?

Partially yes and partially no but in any case that is not connected to my pull request. I don't change any of that code. If too many nodes checkin all at the same time that might cause problems no matter what the config value is set to.