Provide a Swift Metrics backend that exports metrics using OTLP/gRPC

[simonjbeaumont]

This repo currently provides a Swift Tracing backend that exports spans using OTLP, but currently does not provide a backend for Swift Metrics or Swift Log.

I'd like to use this issue to capture and agree a design for how this package can provide a backend for Swift Metrics.

While the Swift Tracing and OTel Tracing APIs are a very close match, there's not such a great impedance match between the Swift Metrics and OTel Metrics APIs. Specifically, the Swift Metrics API is a subset of what can be expressed using OTel Metrics. Additionally the OTel specification is very principled about the API and implementation that language SDK packages should provide. Furthermore, there is already another package in the OpenTelemetry project which is attempting to provide such an SDK for Swift.

We've discussed this offline and, for these reasons, we think the most pragmatic approach is for this package to not try to provide a "full-blown" OTel SDK for metrics, but rather provide a Swift Metrics backend that exports metrics over OTLP/gRPC, such that it can be used with an OTel Collector or other receivers that accept OTLP.

Since the Swift Metrics API the simpler of the two APIs—e.g. there is no choice of aggregation temporality— so it will require simpler backing types to implement. We can even leverage almost identical backing types used in other Swift Metrics backends: notably Swift Prometheus.

We can then provide the mapping from the metric state to the subset of the OTLP datamodel required when exporting the metrics. For example, a Counter will always emit an OTLP Sum with cumulative aggregation temporality (cf. delta).

We will use a principled layered approach similar to the tracing support in this package and decouple the storage types from both the Swift Metrics backend support and the OTLP exporting. For the OTLP exporter we will try to use terms and configuration that map to the OTel specification to the extent it makes sense with the subset of functionality we're providing.

Rough breakdown

• Generate Swift code from Protobuf for OTel metrics (Generate Swift code from Protobuf for OTel metrics #82)
• Add metric store and registry types (Add metric store and registry types #85)
• Provide conformances for a SwiftMetrics backend (Add OTLPMetricsFactory, a Swift Metrics backend #88)
• Add subset of OTLP data model and conversions (Add OTLPMetricsDataModel, OTelInstrument, and OTelMetricProducer #89)
• Add OTelMetricExporter protocol and noop, console, and multiplex exporters (Add OTelMetricExporter protocol and noop, console, and multiplex exporters #94)
• Add OTLPPeriodicExportingMetricsReader and configuration (Add OTelPeriodicExportingMetricsReader and configuration #95)
• Add OTLPGRPCMetricsExporter and configuration (Add OTLPGRPCMetricsExporter and configuration #96)
• Update HTTP server example package with metrics (Update HTTP server example with metrics middleware #90)
• Populate OTLP unit and description using dimensions provided through Swift Metrics API (Extract unit and description from dimensions provided through Swift Metrics API #99)
• Align counter example package with HTTP server example
• Treat Counter and FloatingPointCounter as distinct metrics (Treat Counter and FloatingPointCounter as distinct metrics #104)
• Review and commit to public API shape (Rework public API (still SPI) for metrics #106)
  • struct OTLPMetricsFactory will need to be the point of config.
  • OTelPeriodicExportingMetricsReader will need to take factory and extract its registry internally.
  • nameAndLabelSanitizer should support returning nil to drop.
  • OTLPMetricsFactory configuration values should be moved into their own struct so we can let factory = ....
• Bikeshed final API naming and remove @_spi(Metrics) annotation (Remove @_spi(Metrics) annotation from metrics API #110)
• Add documentation, including the line we're walking between the Swift Metrics API and the OTel specification

Work-in-progress branch

For those that want to follow along as the work progresses, I am maintaining a WIP branch, which I will keep rebasing against main as PRs land, and so will show the current state of outstanding items. This can be used to see how these types compose, including how they are used in the example projects: main...simonjbeaumont:sb/metrics-wip.

Outstanding questions

• Should OTelMetricRegistry be part of the public API, or should we just make it internal and have OTLPMetricsFactory be the only public entry point, as the Swift Metrics backend?
  • Probably not, because (1) Hiding the producer won't allow us to hide much else because of the other layers that we do want to expose, e.g. the exporter and reader, which depend on the same abstractions, e.g. the datamodel; and (2) it would introduce a layering violation between the Swift Metrics factory, which should be to cleanly layer on top a set of composable protocols that make sense in their own right; and (3) by removing this layer, we remove the ability for folks to plug parts of this library together and benefit from the composable building blocks.
• Should the exporting to single OTLP datapoint use the timestamp of measurement (at export time) or the timestamp of the most recent event (time at which it was last changed through the metrics API)?
  • After getting some feedback from potential adopters, this seems fine for now.
• How should we handle unit and description with metrics created through the Swift Metrics API, which currently only provides a set of labels?
  • Extract the values for keys unit and description from the dimensions array Swift Metrics API parameter, if they are present.
• Should we consider floating point and integer counters to be distinct¹?
  • Yes, it seems like it's expected semantics in the OTel specification.

Footnotes

1. https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/api.md#instrument ↩

[slashmo]

the most pragmatic approach is for this package to not try to provide a "full-blown" OTel SDK for metrics, but rather provide a Swift Metrics backend that exports metrics over OTLP/gRPC

+1 on this implementation plan, it's very similar to how I approached tracing support in this library. swift-otel is server-side Swift first, OTel second.

For the OTLP exporter we will try to use terms and configuration that map to the OTel specification to the extent it makes sense with the subset of functionality we're providing.

Also +1 on this, no need to reinvent defaults and/or environment variable names for configuration.

[ktoso]

Thanks for the writeup, that all sounds great :)

Let's have this package be minimal, and we should have a chat with swift-opentelemetry as well at some point to see how we can help them improve the server focus in that implementation 👍

[simonjbeaumont]

OK, I'd like to tackle one of the open questions to unblock progress; this one: How should we handle unit and description with metrics created through the Swift Metrics API, which currently only provides a set of labels?

It's probably clear just from the question, but to spell it out: OTLP data points have a description and unit field. Both of these are considered identifying in the OTel spec. The spec also says that these are optional in the API/SDK. Because of this, we're technically providing compliant OTLP metrics by doing nothing and emitting empty values for these fields in the export, but we need to decide if/how we let users set these.

Given that, at the time of metric recording, Swift Metrics API users only have a set of (String, String) labels, I can see two options:

1. We elevate the semantics of labels with special keys. Specifically, we will use the labels with keys description and unit for the description and unit in the OTLP export respectively.
2. We have some method of configuring this at the time of bootstrap; some sort of dictionary from metric name to these units and descriptions.

I see (2) as a non-starter because the user bootstrapping the metrics system isn't necessarily the user recording the metrics (e.g. if using a dependency that records metrics through the Swift Metrics API), so strikes me to have limited value and present a layering violation.

@slashmo @ktoso @FranzBusch WDYT to (1). Let's park any discussion of whether Swift Metrics API could evolve for the time being.

[ktoso]

Tbh 1) sounds totally reasonable, since otel has some well known fields, and we can match the name... this sounds reasonable. Agree that 2) mixes things up and ("who knows what") as you said.

[FranzBusch]

I agree 1) seems reasonable and if a user puts a description or unit label into the metric then we should just use that for OTEL.

[simonjbeaumont]

@Joannis @ktoso @slashmo and I met and hashed out some answers to the open questions. Concrete actions, which I will add as subtasks to this issue.