[ADDED] option to keep or remove js headers

[ramonberrutti]

Add an option to keep or remove headers from a stream.

This is very useful for source streams where sometimes you don't want to store extra headers or have clients you can't control publishing to your streams.

Signed-off-by: Ramon Berrutti ramonberrutti@gmail.com

[bruth]

Hi @ramonberrutti. What concrete use case drove this proposal? You note sourcing. There are messages being sourced that you either want to add or remove headers?

My concern is that this is technically user-defined processing of the message while being replicated. There is still some scoping and designing to do, but there is a plan to introduce a "message callout" which would enable a generic means for single message transforms.

In conversations, the current scope had not considered sourcing/mirroring, but this will be something worth considering.

My concern is that this could be a slippery slope of adding message processing behavior as part of stream configuration.

[ramonberrutti]

Hi @ramonberrutti. What concrete use case drove this proposal? You note sourcing. There are messages being sourced that you either want to add or remove headers?

My concern is that this is technically user-defined processing of the message while being replicated. There is still some scoping and designing to do, but there is a plan to introduce a "message callout" which would enable a generic means for single message transforms.

In conversations, the current scope had not considered sourcing/mirroring, but this will be something worth considering.

My concern is that this could be a slippery slope of adding message processing behavior as part of stream configuration.

Hi @bruth

I would love "message callout", but that would be impossible in a high-throughput environment.

There are two big cases that I want to solve right now:

• Remove: "Nats-Expected-Last-Subject-Sequence" from the KV store without breaking any other flow (I removed the Expected headers from a source stream in a previous PR, already merged)
• My users have direct access to the KV store (WebSockets), and I want to restrict the headers they can send to the KV or any other subject consumed by the stream.

I agree that this will contain stream logic for message processing.

I'm also updating the PR to support update changes.