Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create attribute.Value type for []byte values #5324

Closed
jmacd opened this issue May 8, 2024 · 4 comments
Closed

Create attribute.Value type for []byte values #5324

jmacd opened this issue May 8, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@jmacd
Copy link
Contributor

jmacd commented May 8, 2024

Problem Statement

When a user has non-utf8 data that they wish to include in their telemetry, they face limited options. It is not valid to use a string representation for all []byte values, because OTLP has a protobuf representation which specifies valid utf8 for strings.

Therefore, when a user has []byte values, they cannot use a string w/o calling e.g., strings.ToValidUTF8() and modify the original data. At first I thought there might be a specification document that forbids OTel APIs from supporting byte slices, but I looked and didn't find anything, so it looks like no spec changes are needed.

Proposed Solution

The OTLP protocol has an AnyValue member for binary data. We have no API-level way to enter this data. Proposing to create Bytes(string, []byte) KeyValue and BytesValue([]byte) Value members in the attribute package.

Alternatives

As discussed in open-telemetry/opentelemetry-specification#3421, if invalid utf8 is allowed into a gRPC-based export pipeline, the protobuf library obligates considering invalid utf8 as an error, which causes whole batches of telemetry to be rejected. So, the alternative is to consider invalid utf8 an error, and require users to validate their inputs.

Prior Art

The Zap logging library has a constructor named zap.Binary(string, []byte) which is exactly what I'd like, except I consider it more Go-idiomatic to name this Bytes().

@jmacd jmacd added the enhancement New feature or request label May 8, 2024
@MrAlias
Copy link
Contributor

MrAlias commented May 8, 2024

Please see #5089. This was already proposed and, as pointed out here, the specification SIG has decided this is not something we want to do.

@jmacd
Copy link
Contributor Author

jmacd commented May 8, 2024

I followed the linked issue to open-telemetry/opentelemetry-specification#3950, and it appears unresolved. If you do not object, I think we should leave this open or close this and re-open #5089. I believe users have a legitimate request, with the zap.Binary method and the presence of an OTLP bytes_value.

@pellared
Copy link
Member

pellared commented May 8, 2024

I believe users have a legitimate request, with the zap.Binary method and the presence of an OTLP bytes_value.

Just for clarification, log attributes do support []byte: https://pkg.go.dev/go.opentelemetry.io/otel/log#Bytes

Bytes is not supported for other signals (tracing, metrics).

@jmacd
Copy link
Contributor Author

jmacd commented May 14, 2024

@pellared Surprising. I guess users can't use the same attributes for logs and spans. I still think the specification should move to allow the byte-valued slices, but am not going to champion this.

@jmacd jmacd closed this as completed May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants