feat: support COSI BucketInfo in replica config

[shanduur]

1. feature: added support for COSI BucketInfo
   • Introduced support for Container Object Storage Interface (COSI) BucketInfo.
   • Added a new method readBucketInfo in the Config struct to read COSI BucketInfo for each database replica.
   • Created a new function parseBucketInfo to handle the parsing of BucketInfo from a specified file.
   • Modified the ReadConfigFile function to call the new readBucketInfo method.
   • The new feature enhances litestream by supporting COSI BucketInfo, enabling more versatile storage configurations in Kubernetes.

Newly introduced code allows specifying new field called bucket-info. Configuration file as this:

dbs:
  - path: /path/to/db
    replicas:
      - bucket-info: /path/to/BucketInfo

Will be expanded based on the contents of the BucketInfo file to something like this:

dbs:
  - path: /path/to/db
    replicas:
      - bucket-info: /path/to/BucketInfo
        region: foo-bar
        bucket: foo-bar
        endpoint: https://foo-bar.example.com
        secret-access-key: EXAMPLE
        access-key-id: EXAMPLE

2. fix: remove deprecated code

   • Removed deprecated code from the codebase.
   • Eliminated obsolete functionality to maintain a clean and up-to-date codebase.
   • This commit improves the maintainability of the code by eliminating deprecated or unnecessary components.

3. fix: removed issues found by staticcheck.

   • Addressed and resolved issues identified by staticcheck.
   • Conducted a static analysis of the code and fixed potential issues, ensuring better code quality.
   • This commit focuses on improving the codebase by eliminating potential sources of bugs or inefficiencies.

4. feature: report errors from stopping/starting replication

   • Enhanced error reporting during the stopping and starting of replication.
   • Improved the logging of errors during these processes to provide more informative messages.
   • This improvement aids in diagnosing issues related to replication, contributing to better system observability and troubleshooting.

---

COSI stands for Container Object Storage Interface. It is a standard developed by the Kubernetes community as part of the SIG-Storage project. COSI focuses specifically on defining an interface for object storage systems within the context of containerized applications and orchestrators like Kubernetes.

The primary goal of COSI is to provide a standardized way for containerized applications to interact with different object storage systems seamlessly. This interface allows developers and operators to use various object storage solutions without requiring changes to the application code. It abstracts the underlying details of the object storage provider, making it easier to switch between different storage backends.

Key components of COSI include:

1. BucketInfo: In the context of litestream's changes, BucketInfo refers to the metadata or configuration information associated with a bucket in an object storage system. It specifies details such as the bucket name, authentication credentials, endpoint, and other relevant settings.

   The BucketInfo is a JSON object inside Secret, that can be mounted in the pod. In the mount path the new file will be created called BucketInfo. The contents should look like this:

   {
       "metadata": {
           "name": "foo-bar",
           "creationTimestamp": null
       },
       "spec": {
           "bucketName": "foo-bar",
           "authenticationType": "KEY", // Alternatively IAM, not feasible in our case
           "secretS3": {
               "endpoint": "s3://foo/bar",
               "region": "foo-bar",
               "accessKeyID": "EXAMPLE",
               "accessSecretKey": "EXAMPLE"
           },
           // Only one of the secrets can be present at the time
           "secretAzure": null,
           // "secretAzure": {
           //     "accessToken": "EXAMPLE",
           //     "expiryTimeStamp": null,
           // },
           // Only one of the protocols should be present at the time
           "protocols": [
               "S3",
               // "Azure",
               // "GCS"
           ]
       }
   }

2. Protocols: COSI supports 3 different protocols for communication with object storage systems - S3 (Simple Storage Service), Azure Blob, GCS (Google Cloud Storage).

3. Authentication Types: COSI supports different authentication mechanisms, such as access keys, secret keys, and various authentication types specific to different object storage providers.

In the context of your litestream changes, it seems that support for parsing COSI BucketInfo is being added, allowing litestream to work with Kubernetes-compliant object storage configurations. This enhances the flexibility of litestream by supporting a wider range of object storage solutions through the COSI standard.

[shanduur]

@benbjohnson I am willing to keep the maintenance of that feature, update API and extend functionality if needed.

[hifi]

I think this is a cool PR, but this switch is not configuring Azure or GCP.

[shanduur]

Yep, it is not. I will do that soon for Azure, not sure about GCP, as there is no GCP secret defined in COSI alpha specification.

[shanduur]

Ah, I don't know if the Azure Blob provides enough info for the litestream to configure replica client - it is only giving the BucketName and AccessToken - this probably is not enough data.