Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker compose db_migration is missing gcp volume #1599

Open
MathijsNL opened this issue Apr 2, 2024 · 2 comments
Open

Docker compose db_migration is missing gcp volume #1599

MathijsNL opened this issue Apr 2, 2024 · 2 comments

Comments

@MathijsNL
Copy link
Contributor

After running a fresh instance the container for the migration throws this error.

 docker logs diffgram-db_migration-1
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
[2024-04-02 20:15:20,695] ERROR data_tools_core_gcp.py Line 42: Error intializing GCP Client
ERROR [shared] Error intializing GCP Client
Traceback (most recent call last):
  File "/app/shared/data_tools_core_gcp.py", line 30, in __init__
    self.gcs = storage.Client(settings.GOOGLE_PROJECT_NAME)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/google/cloud/storage/client.py", line 121, in __init__
    super(Client, self).__init__(
  File "/usr/local/lib/python3.11/site-packages/google/cloud/client.py", line 319, in __init__
    Client.__init__(
  File "/usr/local/lib/python3.11/site-packages/google/cloud/client.py", line 178, in __init__
    credentials, _ = google.auth.default(scopes=scopes)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/google/auth/_default.py", line 459, in default
    credentials, project_id = checker()
                              ^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/google/auth/_default.py", line 221, in _get_explicit_environ_credentials
    credentials, project_id = load_credentials_from_file(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/google/auth/_default.py", line 107, in load_credentials_from_file
    raise exceptions.DefaultCredentialsError(
google.auth.exceptions.DefaultCredentialsError: File /gcp/gcp-service-account.json was not found.

It seems it is missing the volume:

  db_migration:
    image: diffgram/default:${DIFFGRAM_VERSION_TAG}
    env_file:
      - .env
    environment:
      PYTHONPATH: "/app/"
    command: ["/app/db-init.sh", "${DATABASE_HOST}"]
    depends_on:
      db:
        condition: service_healthy
      rabbitmq:
        condition: service_healthy
# The volume is currently missing
    volumes:
    - ${GCP_SERVICE_ACCOUNT_FILE_PATH}:/gcp/gcp-service-account.json

https://github.com/diffgram/diffgram/blob/master/docker-compose.yaml#L92-L103

@oss-maintainer-12
Copy link
Contributor

@MathijsNL Thanks for flagging this.
By chance have you had time to also test if adding that volume fixes it?
This file hasn't change in a while so I'm not sure what would have caused the regression, but if adding the volume works we can simply add it to the file.
I should not that GCP is optional in theory - not sure if that matters for this context

@MathijsNL
Copy link
Contributor Author

The container doesn't throw any errors anymore when I add the volume. But I am also not completely sure why the db migration would need the GCP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants