Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache connections based on the datasource updated timestamp #111

Closed

Conversation

njvrzm
Copy link
Contributor

@njvrzm njvrzm commented Feb 16, 2024

An issue was created for athena-datasource (grafana/athena-datasource#264) where if you rotate AWS credentials and then edit them in the datasource config in grafana, the new credentails aren't used until the datasource is recreated (or the server restarted). This is because database connections there are cached based only on the datasource UID and (if present) the connection arguments.

The grafana-aws-sdk uses a cache mechanism copied from sqlds, and the same issue is present in all sqlds-based (and awsds-based) datasources as well. To address this I've updated sqlds to include the Updated timestamp from the datasource settings in its cache key, and to expose the key mechanism so athena can use it instead of duplicated code.

This does, unfortunately, slightly change the sqlds API - the GetDBFromQuery method now takes backend.DataSourceInstanceSettings instead of a string for the datasource UID. However, as far as I can tell (searching github and google both) this method is only used in grafana-aws-sdk, in one place. Personally I think it's probably okay to keep this at v3 despite the API change, but please let me know if you think we need to bump to v4 instead.

@njvrzm njvrzm added the bug Something isn't working label Feb 16, 2024
@njvrzm njvrzm requested a review from iwysiu February 16, 2024 14:02
@njvrzm njvrzm self-assigned this Feb 16, 2024
@njvrzm njvrzm requested a review from a team as a code owner February 16, 2024 14:02
@njvrzm njvrzm requested review from a team, gabor, yesoreyeram and zoltanbedi and removed request for a team February 16, 2024 14:03
@njvrzm njvrzm linked an issue Feb 16, 2024 that may be closed by this pull request
@njvrzm
Copy link
Contributor Author

njvrzm commented Feb 16, 2024

As it turns out, while this works, it's unnecessarily deep - the problem is actually with how some aws datasources respond to the NewDatasource call from the manager (thanks @gabor for the help figuring this out!). Closing this one.

@njvrzm njvrzm closed this Feb 16, 2024
@shelldandy shelldandy deleted the njvrzm/110-cache_connections_based_on_updated_timestamp branch April 17, 2024 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Cache database connections based on datasource Updated time
1 participant