Spanner: getting the unsupported sampler while keeping the session alive

[harshit-bci]

Client

go-spanner

Environment

GKE

Go Environment

$ go version
1.18.1

Using https://pkg.go.dev/cloud.google.com/go/spanner v1.30.0

Code
go library for Spanner suggests that there is no need to handle session on our own and we are not doing any of the thing on our own. It's the HealthCheckWorker which is doing all of the things on it's own.

Expected behavior

There should not be any error while executing Select 1 .

Actual behavior
Getting Below error
starting span "cloud.google.com/go/spanner.ping": unsupported sampler: 0x9f1c00

Below is the traceback

go.opentelemetry.io/otel.(*delegator).Handle
	/app/vendor/go.opentelemetry.io/otel/handler.go:44
go.opentelemetry.io/otel.Handle
	/app/vendor/go.opentelemetry.io/otel/handler.go:96
go.opentelemetry.io/otel/bridge/opencensus/internal.(*Tracer).StartSpan
	/app/vendor/go.opentelemetry.io/otel/bridge/opencensus/internal/tracer.go:42
go.opencensus.io/trace.StartSpan
	/app/vendor/go.opencensus.io/trace/trace_api.go:56
cloud.google.com/go/spanner.(*session).ping
	/app/vendor/cloud.google.com/go/spanner/session.go:233
cloud.google.com/go/spanner.(*healthChecker).healthCheck
	/app/vendor/cloud.google.com/go/spanner/session.go:1465
cloud.google.com/go/spanner.(*healthChecker).worker
	/app/vendor/cloud.google.com/go/spanner/session.go:1563

Additional context
This issue seems to be fixed here but still we are getting this issue.

[rahul2393]

Thanks @harshit-bci for letting us know about this.
From initial debugging seems like you are using OpenTelemetry and using bridge to convert from opencensus traces,

Error particularly is coming from the bridge code https://github.com/open-telemetry/opentelemetry-go/blob/1f5e6adbf2ca8fdb394ea07b3370506889a10bfa/bridge/opencensus/internal/oc2otel/tracer_start_options.go#L43, Let me try to find the fix to support OpenTelemetry in Spanner.

We already have a open ticket to support OpenTelemetry for traces here #4237.

[rahul2393]

Back to your question on handling sessions, Go client library is handling the sessions fine in your application, its just not able to trace a specific method using OpenTelemetry, I think the error reported in issue is safe to ignore and should not be impacting any database operations.

[harshit-bci]

@rahul2393 By when this can be resolved ? The ticket #4237 (that you mentioned) is already open from last 5 months.

[rahul2393]

@harshit-bci The issue is happening because of OpenTelemetry limitation here https://github.com/open-telemetry/opentelemetry-go/tree/main/bridge/opencensus#incompatibilities to support samplers.

While we are looking into possible solutions out there but it would make more sense if you open a ticket here https://github.com/open-telemetry/opentelemetry-go/issues/new?assignees=&labels=bug&template=bug_report.md&title= because we don't support OpenTelemetry in Go Client libraries yet.

[rahul2393]

This fix seems like dupe of #4237 so I will close out this issue. The implementations for most OpenTelemetry services are not yet stable. This is especially true in Go where tracing is currently stable, metrics are alpha, and logging does not exist. As the project matures over time we will continue to evaluate if it is right to integrate with it.

Source: https://opentelemetry.io/docs/go/

[vvakame]

I met this issue too.
I got another additional error, failed to export to Google Cloud Trace: rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8 by google cloud exporter.
I don't know this issue related to my another error. but if this error is resolved and StartSpan does not return an error, there is less to think about.