feat(processor): preserve auth in batch processor

[grandwizard28]

Description

This PR preserves the authData in the client context after the batch processor. The idea is that the auth data can be used after the batch processor in subsequent processors and exporters.

Testing

• Current unit tests modified to include the context
• Need help with adding unit tests which choose auth is preserved.

[mx-psi]

@grandwizard28 Thanks for the PR! Are there any related issues to this?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.78%. Comparing base (4a58092) to head (45f83ea).

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #10002   +/-   ##
=======================================
  Coverage   91.77%   91.78%           
=======================================
  Files         360      360           
  Lines       16641    16643    +2     
=======================================
+ Hits        15273    15275    +2     
  Misses       1041     1041           
  Partials      327      327           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[grandwizard28]

Hey @mx-psi,
There are no current issues related to this. The background is while using any auth extension, we generally want to propagate some data (for example user_id) across processors and exporters but the batch processor is dropping the clientInfo.Auth struct.

I thought this would make a nice edition to the core since the batch processor is widely used.

Do let me know your thoughts on the above hypothesis :)

[grandwizard28]

Hey @jpkrohling @mx-psi,
Any thoughts here?

[jpkrohling]

I'm not sure this is accurate. I believe this feature exists already, and was implemented by @jmacd on #7578. If that's not working for some reason, I'd be interested in hearing about that in an issue.

[jpkrohling]

The context at this level doesn't have auth data, as newBatchProcessor isn't called in response to incoming data, but during the startup of the collector.

[grandwizard28]

Understood! The idea is to propogate the existing context in all shards created.

The newShard function is the same regardless of whether we want to start at startup with a single shard or start multiple shards as per the metadata. I figured we just need to amend the signature of newShard to give it an existing (if present) AuthData.

Is my understanding fair?

[grandwizard28]

If you look at the exportCtx @jpkrohling, it looks like the following:

exportCtx := client.NewContext(context.Background(), client.Info{
		Metadata: client.NewMetadata(md),
	})

There is no AuthData preservation happening here at all. We are creating a new context altogether.

Do you want me to still create an issue?

[jpkrohling]

No, the code you are looking for is here:

opentelemetry-collector/processor/batchprocessor/batch_processor.go

Line 326 in 7855bf2

b, loaded = mb.batchers.LoadOrStore(aset, mb.newShard(md))

This will create a new shard based on the context for that is available in the consume function, which does have client metadata.

[grandwizard28]

Hey @jpkrohling,
Created an issue for this: #10093