Release/0.1.13

Contents

New features

• Support setup.py and requirements-dev.txt in Cloud Run Dockerfile
• Retrieve credentials from Google Cloud Secret Manager and inject into environment in Runner.run
• Add ability to retrieve and update cloud files via the Datafile.download or Datafile.open methods
• Allow cloud file attributes to be updated via Datafile.to_cloud method
• Allow instantiation of TagSets from JSON-encoded lists

Breaking changes

• Raise error if the datasets of the input manifest passed to Service.ask aren't all cloud-based

Fixes

• Fix Dataset construction from serialised form in Manifest
• Fix Datafile construction from serialised form in Dataset
• Fix Datafile.deserialise
• Adjust usages of tempfile.NamedTemporaryFile to also work on Windows
• Add timeout and retry to Service.answer
• Add retry to Service.wait_for_answer
• Add 60 second timeout for answering question in Cloud Run deployment
• Use correct environment variable for service ID in Cloud Run Dockerfile
• Set _last_modified, size_bytes, and _hash_value to null values if a Datafile representing a cloud file is instantiated for a hypothetical cloud location (i.e. not synced to a cloud file at that point in time)
• Allow Dataset.get_file_sequence use with no filter

Dependencies

• Use new twined version that supports validation of credentials strand
• Use newest version of gcp-storage-emulator

Minor improvements

• Make path a positional argument of Datafile
• Move gunicorn requirement into octue requirements
• Raise warning instead of error if Google Cloud credentials environment variable is not found and return None as credentials
• Move cloud code into new cloud subpackage
• Raise TimeoutError in Service.wait_for_answer if no response is received by end of retries
• Only look for deployment_configuration.json file in docker container /app directory
• Ensure deployment_configuration.json file is always loaded correctly in docker container
• Pass credentials strand into Runner instance in Cloud Run deployment
• Add name attribute to Identifiable mixin
• Add Google Cloud metadata to Datafile serialisation
• Add deserialise method to Datafile
• Add ability to add metadata to a Datafile instantiated from a regular cloud file
• Use CRC32C hash value from Google Cloud when instantiating a Datafile from the cloud
• Add ability to name Datafiles
• Add ability to check whether a Datafile, all Datafiles in a Dataset, or all Datasets in a Manifest are located in Google Cloud
• Use Datafile.deserialise when instantiating a Dataset from a dictionary
• Add representation to GCPPubSubBackend
• Load credentials strand JSON in Runner initialisation
• Add location searched to message of error raised when app module can't be found in Runner.run
• Ignore E203 flake8 warning

Testing

• Remove subjective Service test test_serve_with_timeout
• Use temporary file rather than temporary directory for tests where possible
• Test Dataset.deserialise

Quality Checklist

• New features are fully tested (No matter how much Coverage Karma you have)

Coverage Karma

• If your PR decreases test coverage, do you feel you have built enough Coverage Karma* to justify it?