Skip to content

Add Google Cloud Storage support; deprecate python < 3.8

Compare
Choose a tag to compare
@cortadocodes cortadocodes released this 10 Mar 16:19
· 3244 commits to main since this release
aa00826

Contents

New Features

  • Add GoogleCloudStorageClient
  • Write manifest, its datasets, and its datafiles to cloud in Analysis.finalise (#96)
  • Closes #84 - add auto tag and release workflow
  • Allow Google Cloud storage blobs to be represented by Pathable
  • Add Datafile, Dataset, and Manifest to_cloud and from_cloud methods
  • Allow regular GCP files to be represented as Datafiles

Minor fixes and improvements

  • Add cloud storage emulator once for all tests
  • Add disk usage and file age utilities
  • Allow Datasets to have custom names
  • Add storage.path module akin to os.path but for Google Cloud Storage paths
  • Allow Hashables' hash values to be set
  • Pass GCP project and bucket names to tests from environment (#93)
  • Add ability to delete topic and subscription when a Service has finished serving
  • Facilitate graceful exit for serving Services on KeyboardInterrupt
  • Use latest versions of flake8, isort, and black in pre-commit and across all files (#87)
  • Fix CI test skipping flag
  • Fix documentation links (#92)

Breaking changes

  • Remove testing and explicit support for python3.6 and python3.7
  • Remove base_from from Pathable and replace with more transparent method
  • Rename Datafile.posix_timestamp to Datafile.timestamp and remove default value
  • Make Datafile.last_modified private
  • Rename persistence subpackage to storage

Testing

  • Test that children can question their own children as part of answering a question
  • Close #94 - delete topics and subscriptions at the end of each test
  • Remove timeouts from tests and replace with thread executor shutdown upon test pass, meaning that tests that connect to Google Pub/Sub won't fail because the connection is slower than expected