fdw: Add foreign data wrappers to read snapshots on s3, gcs or azure

[mfussenegger]

Problem Statement

Keeping all data forever in CrateDB can get expensive.
Deleting the data isn't an option, because it might still be needed, but it is not queried often.

Having an option to use cheaper storage option at the expense of query performance would be nice.

Possible Solutions

Backups already exist in the form of snapshots. These snapshots could be exposed via a foreign table wrapper/foreign table to be able to query them ad-hoc.

Advantages:

• No need for additional data exports
• Infrastructure is already setup if there are backups
• Snapshots contain segments file, we might be able to use the Lucene structures to do filters more efficiently than with alternative file formats.
• Full control over optimization possiblities, as we control the format.

Downsides:

• Snapshot format compatibility with major version updates(?)
• Others?

Considered Alternatives

• fdw: Add foreign data wrapper for parquet files (read-only) #15721

Technical constraints

• Should be implemented outside the server module and the hardcoded map in ForeignDataWrappers should be replaced via a service loader. This relates to fdw: Expose plugin interface to add additional foreign data wrappers #15720 - except that it can be completely undocumented/private, similar to other service loaders like for Functions

Initial Scope (estimate is only for this)

• Simple but slow version; No caching of downloaded data; Always reads from remote as neeeded; No attempts at operation push-down or partial result retrieval.

Follow up (for later dedicated issues, not included in the first implementation)

• Download only required data to minimize traffic; And maybe add cache for the remote data to avoid repeated downloads
• Index utilization; Retrieve metadata/index files, then only fetch required documents.