django-gcloud-storage

Django storage module implementation for Google Cloud Storage using the google-cloud-storage python module by Google.

Notice: alpha release

Please keep in mind that this version is not yet used in any production application (as far as I know of) and thus is an alpha release, even though fully tested! Any kind of feedback is greatly appreciated.

Installation

Install django-gcloud-storage:

pip install django-gcloud-storage

Create a GCS service account JSON keyfile and a bucket for your application. Check the documentation of Google Cloud Platform for more details:

https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating

https://cloud.google.com/iam/docs/creating-managing-service-account-keys#creating

Update your Django settings and use it like any other Django storage module:

DEFAULT_FILE_STORAGE = 'django_gcloud_storage.DjangoGCloudStorage'

GCS_PROJECT = "django-gcloud-storage"
GCS_BUCKET = "django-gcloud-storage-bucket"
GCS_CREDENTIALS_FILE_PATH = "/path/to/gcs-credentials.json"

Features

• Fully tested on Python 3.7 - 3.10, PyPy 3.7-7.3.7 and PyPy 3.8-7.3.7 with Django 2.2, 3.2 and 4.0
• Files are locally downloaded as SpooledTemporaryFile objects to avoid memory abuse
• Changed files will automatically be reuploaded to GCS when closed

Caveats

• Files must be fully downloaded to be accessed and fully uploaded when changed
• Everytime a file is opened via the storage module, it will be downloaded again
• (development) Most tests need access to Google Cloud Storage

Unsigned URLS

The module generates signed urls by default. This requires calls to storage API which might take some time if you need to return several objects at a time. You can generate unsigned urls using the following setting:

GCS_USE_UNSIGNED_URLS = True

Keep in mind you might need to set the default object permission to public for the unsigned urls to work.

Contributing

Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.

Report Bugs

Report bugs at https://github.com/strayer/django-gcloud-storage/issues.

If you are reporting a bug, please include:

• Your operating system name and version.
• Any details about your local setup that might be helpful in troubleshooting.
• Detailed steps to reproduce the bug.

Fix Bugs

Look through the GitHub issues for bugs. Anything tagged with "bug" is open to whoever wants to implement it.

Implement Features

Look through the GitHub issues for features. Anything tagged with "feature" is open to whoever wants to implement it.

Write Documentation

django-gcloud-storage could always use more documentation, whether as part of the official django-gcloud-storage docs, in docstrings, or even on the web in blog posts, articles, and such.

Pull Request Guidelines

Before you submit a pull request, check that it meets these guidelines:

1. The pull request should include tests.
2. The pull request should work for all supported Python and Django versions (see above). Make sure that the tests pass.

Running Tests

Warning: Most of the tests require a GCS project and will do API requests that may end up costing you money!

You can run the test suite either in a virtualenv with py.test or with tox - both require a valid service account JSON keyfile called test-credentials.json in the project root. The GCS project name will be provided via a command argument.

The tests will create and (hopefully) remove buckets on their own. To be safe, check if there are any leftover buckets in your GCS project after running the tests!

source <YOURVIRTUALENV>/bin/activate
(myenv) $ pip install -r requirements-test.txt
(myenv) $ pip install -e .
(myenv) $ py.test --gcs-project-name="project-name"

or

$ tox -- --gcs-project-name="project-name"

Credits

Inspired by:

• django-storages

Tools (partly) used in rendering this package:

• Cookiecutter
• cookiecutter-djangopackage