Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Work around Vault DB static creds TTL rollover bug #730

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

benashz
Copy link
Collaborator

@benashz benashz commented May 8, 2024

When syncing database static credentials role configured with scheduled rotation, the TTL is incorrectly rolled over within the same rotation period. Since, VSO relies on the TTL for its sync scheduling, an invalid TTL results in syncing stale credentials.

This fix, attempts to detect the TTL rollover bug, and ensure that current rotated creds are properly synced.

@benashz benashz requested a review from a team as a code owner May 8, 2024 14:45
@benashz benashz force-pushed the VAULT-26529/vds-work-around-scheduled-static-creds-ttl-rollover branch from 498d725 to 0115e07 Compare May 8, 2024 14:48
When syncing database static credentials role configured with scheduled
rotation, the TTL is incorrectly rolled over within the same rotation
period. Since, VSO relies on the TTL for its sync scheduling, an invalid
TTL results in syncing stale credentials.

This fix, attempts to detect the TTL rollover bug, and ensure that
current rotated creds are properly synced.
@benashz benashz force-pushed the VAULT-26529/vds-work-around-scheduled-static-creds-ttl-rollover branch from 0115e07 to 5d6bb40 Compare May 8, 2024 14:52
@@ -348,6 +348,7 @@ func Test_resourceReferenceCache(t *testing.T) {
}

func TestSyncRegistry(t *testing.T) {
t.Skip()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest mentioning why this is now skipped

Comment on lines +492 to +493
// considered to be reliable to the TTL roll-over bug that might exist in the database
// secrets engine.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this bug going to be fixed on the Vault side? i.e. wondering if we need to support a buggy Vault version if it's going to be fixed in the next patch release of Vault?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that is a great question. We will be fixing the issue in Vault, just no ETA yet. I think it's okay to provide a best effort workaround for those users affected, especially since this workaround is to address a support escalation.

}

if v, ok := data["rotation_schedule"]; ok && v != nil {
if schedule, ok := v.(string); ok && v != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think v != nil is already known from the previous if?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 145b47a

)

bo := backoff.NewExponentialBackOff(
// typically the rotation period is 5s, so it should be safe to double that.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 5s is the minimum rotation_period? Not necessarily typical?

Copy link
Collaborator Author

@benashz benashz May 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 145b47a

@benashz benashz force-pushed the VAULT-26529/vds-work-around-scheduled-static-creds-ttl-rollover branch from fa31cba to 145b47a Compare May 22, 2024 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants