Allow new datafiles to be created in the cloud without a local file existing first

Contents

Enhancements

• Allow new datafiles to be created in cloud without local file

Fix log handling

Contents

Enhancements

• Improve Service log messages
• Keep full module path in logging context when using analysis logger
• Demote storage client log messages to DEBUG level

Fixes

• Fix and improve apply_log_handler function
• Apply log handler to the root logger in top-level __init__ module
• Stop duplication of timestamp in Google Cloud Run logs by applying different handler/formatter if PLATFORM environment variable is present and equal to GOOGLE_CLOUD_RUN
• Add COMPUTE_PLATFORM=GOOGLE_CLOUD_RUN environment variable to docker image
• Remove applying of log handler from incorrect places
• Remove extra logger in runner module
• Stop analysis logger propagating to the root logger and repeating log messages

Reversions

• Remove show-twined-logs option from CLI

Testing

• Silence GCP Storage emulator logs below ERROR level
• Ensure unmocked Subscription test always deletes topic and subscription
• Remove extra logger from tests.base
• Avoid irrelevant exception in Cloud Run tests

Operations

• Remove unnecessary preceding underscores from ARGs in Dockerfile

Use question UUID as analysis ID

Summary

Use service question UUIDs as analysis IDs to allow log messages for a Google Cloud Run service to be filtered for a given question. Also improve the presentation of error messages forwarded from the responding service to the asking service.

Contents

Enhancements

• Use service question UUIDs as analysis IDs
• Dynamically create exception class in Service._raise_exception_from_responder if the received type name is unknown to the asker

Minor improvements

• Use a string UUID for question IDs rather than an integer UUID

Allow non-JSON-encoded items in GCS metadata

Contents

Fixes

• Allow non-JSON-encoded items in GCS metadata (this means a Datafile can be instantiated from any GCS object with arbitrary metadata rather than raising an error)

Remove `sequence` and `cluster` attributes from Datafile

Contents

Breaking changes

• Remove the sequence and cluster attributes from Datafile (they were simply tags with unnecessary special treatment)

Refactoring

• Replace Dataset.get_file_sequence with added functionality in FilterContainer.order_by

Operations

• Ensure breaking change commits increment minor version number in mkver.conf

Handle errors in responding service and raise in asking service

Contents

Enhancements

• Log purely service-caused exceptions in Google Cloud Run without raising them, always returning a 204 response unless there is an error in Google Cloud Run itself
• Forward service-caused exceptions back to local asker and raise them there, separating the local traceback from the remote traceback

Refactoring

• Use built-in datetime POSIX time methods instead of custom-written functions

Testing

• Test Cloud Run answer_question function locally (it's already tested remotely in the deployment tests but this doesn't show up in the test coverage)

Fix: Remove disparity between Datafile instantiation methods

Contents

Refactoring/fixes

• Combine Datafile.__init__ with Datafile.from_cloud for simpler and more robust interface
• Carry out functionality of former Datafile.from_cloud method in Datafile.__init__ if the path parameter is a cloud path

Reversions

• Remove ability to override cloud metadata on locally instantiated Datafile during instantiation

Use conventional-commits repo for CI checks

Contents

Operations

• Use conventional-commits repo for CI checks

Ensure tests always receive test result modifier on Windows

Contents

Testing

• Ensure tests always receive test result modifier on Windows

Fix: Ensure Service.wait_for_answer timeout value is respected

Contents

Enhancements

• Add more configuration options to Subscription
• Make serving services' subscriptions never expire

Fixes

• Ensure Service.wait_for_answer timeout parameter is respected
• Delete answer topic and subscription after service wait timeout

Operations

• Use LAST_RELEASE mode when compiling release notes

Dependencies

• Use google-cloud-pubsub>=2.5.0