You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a user has non-utf8 data that they wish to include in their telemetry, they face limited options. It is not valid to use a string representation for all []byte values, because OTLP has a protobuf representation which specifies valid utf8 for strings.
Therefore, when a user has []byte values, they cannot use a string w/o calling e.g., strings.ToValidUTF8() and modify the original data. At first I thought there might be a specification document that forbids OTel APIs from supporting byte slices, but I looked and didn't find anything, so it looks like no spec changes are needed.
Proposed Solution
The OTLP protocol has an AnyValue member for binary data. We have no API-level way to enter this data. Proposing to create Bytes(string, []byte) KeyValue and BytesValue([]byte) Value members in the attribute package.
Alternatives
As discussed in open-telemetry/opentelemetry-specification#3421, if invalid utf8 is allowed into a gRPC-based export pipeline, the protobuf library obligates considering invalid utf8 as an error, which causes whole batches of telemetry to be rejected. So, the alternative is to consider invalid utf8 an error, and require users to validate their inputs.
Prior Art
The Zap logging library has a constructor named zap.Binary(string, []byte) which is exactly what I'd like, except I consider it more Go-idiomatic to name this Bytes().
The text was updated successfully, but these errors were encountered:
I followed the linked issue to open-telemetry/opentelemetry-specification#3950, and it appears unresolved. If you do not object, I think we should leave this open or close this and re-open #5089. I believe users have a legitimate request, with the zap.Binary method and the presence of an OTLP bytes_value.
@pellared Surprising. I guess users can't use the same attributes for logs and spans. I still think the specification should move to allow the byte-valued slices, but am not going to champion this.
Problem Statement
When a user has non-utf8 data that they wish to include in their telemetry, they face limited options. It is not valid to use a
string
representation for all[]byte
values, because OTLP has a protobuf representation which specifies valid utf8 for strings.Therefore, when a user has []byte values, they cannot use a string w/o calling e.g.,
strings.ToValidUTF8()
and modify the original data. At first I thought there might be a specification document that forbids OTel APIs from supporting byte slices, but I looked and didn't find anything, so it looks like no spec changes are needed.Proposed Solution
The OTLP protocol has an AnyValue member for binary data. We have no API-level way to enter this data. Proposing to create
Bytes(string, []byte) KeyValue
andBytesValue([]byte) Value
members in theattribute
package.Alternatives
As discussed in open-telemetry/opentelemetry-specification#3421, if invalid utf8 is allowed into a gRPC-based export pipeline, the protobuf library obligates considering invalid utf8 as an error, which causes whole batches of telemetry to be rejected. So, the alternative is to consider invalid utf8 an error, and require users to validate their inputs.
Prior Art
The Zap logging library has a constructor named
zap.Binary(string, []byte)
which is exactly what I'd like, except I consider it more Go-idiomatic to name thisBytes()
.The text was updated successfully, but these errors were encountered: