Google: Harmonize Google plugin auth #39907
Replies: 2 comments
-
Thanks, @sunker for creating this, think this is a great proposal and will help a lot in maintaining google data sources. cc @ryantxu - something we talked about not that long ago ^^ |
Beta Was this translation helpful? Give feedback.
0 replies
-
Done in #44537 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
What would you like to be added?:
I would like all Google plugins to have the same authentication capabilities and the code and documentation for this to be handled in one single place.
Why?
The authentication methods
JWT File
andGCE Default Service Account
are currently only reusable for core plugins and for external plugins that doesn't have a backend. Also, providing shared packages for handling Google plugin auth will make it easier for us and the community to build new Google plugins.Existing Google plugins
Currently, Grafana maintain three Google plugins. The CloudMonitoring datasource which is a part of core Grafana plugin maintained by the cloud data sources squad, the BigQuery plugin which is an external plugin maintained by the BI squad and the google sheets plugin plugin which is also an external plugin that is being maintained by the community squad.
Authentication methods
JWT Key File
A Service Accounts can be used for accessing APIs in a certain GCP project. It's possible to generate a JWT key file that is associated with the service account. The credentials in this file can then be used to generate an OAuth 2.0 token that can be used in API calls to google APIs.
The code for handling JWT key file auth is located in the data source proxy. This authentication method is currently available in CloudMonitoring and BigQuery. Google sheets also has support for JWT key file auth, however it's not using the auth proxy. The google sheets plugin uses the google go sdk for interacting with the service which is fine, however the token generated for the JWT file is not being cached so there's room for improvements here.
GCE Default Service Account
Virtual machines in Google Compute Engine can associated with a Service Account. If Grafana is running on a GCE instance, Grafana can obtain an OAuth2 token for the Service Account using the GCE meta data server api. This token can then be used to access google APIs, given the service account has the associated permissions.
The code for handling GCE auth is located in the data source proxy. This authentication method is currently available in CloudMonitoring and BigQuery, but should be a valid auth method also for Google Sheets (grafana/google-sheets-datasource#144).
API Key
This authentication option can be used for accessing public spreadsheets in Google Sheets. It's likely not a valid authentication method for other plugins, so this probably doesn't belong in a shared SDK.
The data source proxy
The data source proxy in core Grafana provides different methods of authenticating requests to a data source api by specifying routes in the plugin.json file.
Limitations of the data source proxy
The initial idea of this proxy was to provide authentication to plugins that doesn't have a backend. These authentication methods might also be usable for plugins that has a backend - like in the QueryDataHandler for example. For core plugins, this is possible to achieve by using the ApplyProxy function in the proxy. This function is however not available to external plugins, so currently there's no way of providing
JWT Key file auth
orGCE auth
in the backend for these plugins.Suggested solution
The logic for providing tokens should not be in the plugin proxy, but in an go package in a separate repository. This go package should house logic for retrieving, caching and refreshing tokens for JWT file auth and GCE auth. It should expose methods for retrieving these tokens and take scopes as an argument. It should also expose an auth middleware (similar to this one) that can be registered in the http clients for backend plugins. In order to not introduce breaking changes, the data source proxy should still support gce auth and jwt auth. However they should use token providers from the shared go package.
The work can be split up into multiple tasks and separate issues can be created for most of them
grafana-google-sdk
- DONE. See https://github.com/grafana/grafana-google-sdk-gografana-google-sdk
. Google: Move google plugin token provider logic to shared SDK #39950This work will be very similar to what has been done/is ongoing for Azure (see this epic) and for AWS.
Feedback appreciated @grafana/cloud-datasources @grafana/grafana-bi-squad
Beta Was this translation helpful? Give feedback.
All reactions