Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting message timestamp #5310

Open
GeorgeGkinis opened this issue Apr 14, 2024 · 3 comments
Open

Allow setting message timestamp #5310

GeorgeGkinis opened this issue Apr 14, 2024 · 3 comments
Labels
proposal Enhancement idea or proposal

Comments

@GeorgeGkinis
Copy link

Proposed change

Allow setting message timestamp when pushing to a jetstream.

Use case

From my understanding so far, setting the timestamp of a message being pushed to a jetstream is not currently possible and the message gets its timestamp based on the receive time.

I would like to be able to set the timestamp when pushing to a jetstream, so I could upload historical event data (i.e. from csv using benthos), setting the timestamp based on the original event timestamp and then use replay policy: original.

If I do this now, then replaying happens in the ingestion speed when pushing to the jetstream, not the original speed.

It would be even more awesome if the messages still get ordered in the jetstream based on their timestamp, even if they are pushed out-of-order.

This is possible with influxdb (which is a DB and not a message distributor like nats.)

Contribution

Yes, if qualified.

@GeorgeGkinis GeorgeGkinis added the proposal Enhancement idea or proposal label Apr 14, 2024
@kfollesdal
Copy link

This would also be nice feature to have if you make your own "tired storage"

@Zetanova
Copy link

Zetanova commented May 5, 2024

I tried this now over the header Nats-Time-Stamp but with bad results at least on nats 2.9.20

The server persist the header but still takes its own received timestamp
and the c# NATS.Client 1.1.4 does silently throws an parse exception on msg.Time.

I thin the current only workaround is use a custom header for the timestamp.

@derekcollison
Copy link
Member

This is not supported at this time but is being considered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
proposal Enhancement idea or proposal
Projects
None yet
Development

No branches or pull requests

4 participants