Release v2.11.0-preview.2

Changelog

Note: this preview release incorporates the cumulative changes in the 2.10.x series since version 2.10.12.

Go Version

• 1.22.2 (#5300)

Added

TLS

• Add CA search functionality for the Windows cert store (#5115)

JetStream

• Add TPM-driven filestore encryption for Windows (#5273)
• Support StartTime for get/direct message requests (#5158)

Monitoring

• Add logical account name to accstatz endpoint (#5173)

MQTT

• Add Sparkplug B compatibility (#5150)

WebSocket

• Support custom response headers in configuration (#5230) Thanks to @ramonberrutti for the contribution and @mcosta74 for the proposal!

Fixed

JetStream

• Fix consumer resume on empty/zero PauseUntil value (#5164) Thanks to @MauriceVanVeen for the contribution!

Complete Changes

v2.11.0-preview.1...v2.11.0-preview.2

Release v2.10.14

Changelog

(Note there was no 2.10.13 version 🙂)

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.21.9 (#5300)

Dependencies

• github.com/nats-io/nats.go v1.34.1 (#5271)
• golang.org/x/crypto v0.22.0 (#5283)

Improved

Auth

• Improve clone behavior to prevent unintended references (#5246) Thanks to Trail Of Bits for the report!
• Apply constant-time evaluation of non-bcrypt passwords (#5247) Thanks to Trail Of Bits for the report!

JetStream

• Reduce lock contention when looking up stream metadata (#5223)
• Optimize matching a subject when applying per subject message limits (#5228)
• Optimize waiting queue for pull consumers to reduce excessive memory and GC pressure (#5233)
• Improve error handling in filestore to prevent duplicate nonces being used and ignored errors (#5248) Thanks to Trail Of Bits for the report!
• Improve interest and workqueue state tracking to prevent stranded messages during concurrent consumer acks and stream deletes (#5270)
• Introduce store method to push down and optimize multi-filter subject matching used by consumers (#5274) Thanks to @svenfoo for the report!
• Various improvements and fixes for clustered interest-based streams and associated consumers (#5287)
• Return errors and/or adding logging for rare filestore conditions (#5298)
• When explicitly syncing to the filesystem, hold the message block lock to prevent possible downstream corruption (#5301, #5303)

Fixed

OS

• Fix for race checkptr panic on macOS/Darwin on Go 1.22 (#5265)

Connections

• Address possible memory leak due to connections not be released (#5244) Thanks to @davidzhao for the report!

JetStream

• Fix incorrect subject overlapping checks that could lead to multiple consumers or streams bound to the same subjects (#5224)
• Improve situations that could result in orphan messages in streams (#5227)
• Protect against corrupt message block when doing indexing (#5238) Thanks to @kylemcc for the report!
• Fix consumer config check of max deliver when backoff is set (#5242)
• Ignore Nats-Expected-* headers from source stream (#5256) Thanks to @ramonberrutti for the report and contribution!
• Add missing check that could result an extended purge or compact to fail in memory-based streams (#5264)
• Fix issue that could result in skipping valid messages when loading them from the filestore (#5266)
• Use cluster-scoped lock when processing a leader change (#5267)
• Fix missing unlocks in filestore and streams in certain error conditions (#5276) Thanks to Trail Of Bits for the report!
• Ensure lock is held for the duration of a filestore truncate (#5279)
• Fix race condition when checking for the stream interest state (#5290)
• Ensure dangling NRG directories are cleaned up when the in-memory stream/consumer are deleted (#5291)
• Perform a standard stream purge when ack floor is higher than the last known state (#5293)
• Handle concurrent creation of workqueue consumers that could result in overlapping interest (#5295) Thanks to @LautaroJayat for the report!
• Fix possible case of multiple deliveries of the same message that cause the delivery count decreasing (#5305)

Monitoring

• Fix JSZ account filtering behavior when requesting stream details (#5229)

OCSP

• Fix stapling during gateway reconnect and configuration reload (#5208)

Chores

• Fix incorrect function names in comments (#5237, #5289) Thanks to @depthlending and @needsure for the contributions!
• Improve workqueue stream sourcing tests (#5112)
• Speed-up tests and fixup lint reports (#5258)
• Improve hardened systemd configuration example (#5272)
• Refactor LoadNextMsgMulti store tests to guard against drift (#5275, #5277)
• Rename orphan messages and replicas test cases (#5292)

Complete Changes

v2.10.12...v2.10.14

Release v2.10.12

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.21.8 (#5175)

Dependencies

• github.com/klauspost/compress v1.17.7 (#5129)
• github.com/nats-io/nats.go v1.33.1 (#5104)
• golang.org/x/crypto v0.21.0 (#5199)
• golang.org/x/sys v0.18.0 (#5198)
• github.com/nats-io/jwt/v2 v2.5.5

Improved

General

• Optimize detecting long subjects with wildcards (#5102)
• Refactor getHeader to improve bounds checking (#5135)

JetStream

• Switch to use fmt.Appendf to reduce a few allocations (#5100)
• Write Raft peer state and term/vote inline (#5151, #5178)
• Improve term management when handling Raft vote responses (#5149)
• Ensure Raft applied index is only updated by upper layer (#5152)
• Add I/O gate for message block writeAt syscall (#5193)
• Lower the minimum expiry threshold to 250ms (#5206)

Fixed

General

• Fix randomisation for queue subscriptions on 32-bit platforms which could cause a panic (#5169) Thanks to @jeremylowery for the report!
• Stree not matching when partial parts were compared to long fragments (#5177)

Gateway

• Fix sending empty reply on gateway RMSG (#5192) Thanks to @n-holmstedt for the report!

Leafnodes

• Fix loop detection on daisy-chained leafnodes (#5126)
• Make sure to not remove account mappings that just had their value changed (#5132, #5103)

JetStream

• Fix sending Consumer Deleted on peer remove (#5111, #5160)
• Fix memory leak during compaction within memory store (#5116) Thanks to @stefanLeo for the report and contribution!
• Updating consumer config fails to check OptStartTime properly (#5127) Thanks to @thed0ct0r for the contribution!
• Slow ack for snapshots could cause slow consumer and client disconnect (#5144)
• Fix for a test flapper with consumer expire frequency change (#5155)
• Fix a potential drift that could occur when assigning last sequences to published messages on clustered streams (#5179)
• Fix data race when capturing last sequence on clustered streams (#5182)
• Fix lock inversion when tracking last sequence failures on clustered streams (#5183)
• Revert an earlier change around the Raft handling of stepdowns due to stream move performance (#5200)
• Make sure not to commit a replicated ack when the consumer is closed (#5196)
• Check the Raft layer stream state once recovery is complete (#5196)

OCSP

• Prefer a POST method to the OCSP server, falling back to GET (#5109)
• Fixed a race condition that could affect OCSP stapling during server reloads (#5207)

Complete Changes

v2.10.11...v2.10.12

Release v2.9.25

Changelog

Go Version

• 1.20.14

Improved

JetStream

• In lame duck mode, shutdown JetStream at the start to signal transfer of leadership if the leader (backported from #4579)
• Prevent processing of consumer assignments after JetStream shutdown occurs (backported from #4625)

Fixed

JetStream

• Fix possible stream assignment race condition (backported from #4589)

Complete Changes

v2.9.23...v2.9.24

Release v2.11.0-preview.1

📣 Notice

This is the first preview release of 2.11.0! The motivation is to release new features and changes during the development lifecycle. This release is not feature complete for what the final 2.11.0 release will contain. This should only be used in development environments and not production. Additional preview releases will become available in subsequent weeks. Please check it out and give us feedback!

Preview Docker images are available under the synadia/nats-server Docker repo, not the official nats Docker repo.

Changelog

Go Version

• 1.22.0 (#5133)

Added

Core

• Distributed pub/sub message tracing including OpenTelemetry support (#5014, #5057, #5078, #5121)
  • Refer to the ADR for more details

JetStream

• Support for batched direct-gets requests from streams (#4981, #5036)
  • Refer to the ADR for more details
• Add support for pausing/resuming consumers (#5066)
• Support for batched direct multi-get requests from streams (#5107)
  • Refer to the (in-progress) ADR for more details

Changed

MQTT

• Change retained messages to use headers for metadata rather and JSON-encoding the message (#5050)
  • Note this is currently a backwards incompatible change with 2.10.x servers.

Improved

General

• Track recovery from slow consumer events in routes (#5101)

OCSP

• Make OCSP request with POST method (#5109)

JetStream

• Prevent potential degradation of publish latency over time (#5116) Thanks to @stefanLeo for contributing the issue and the improvement!

Fixed

JetStream

• Add explicit structs for the account claims update response payload (#4942)
• Acking a redelivered msg with more pending outstanding that should move the ack floor (#5008)

MQTT

• Header length calculation for stored retained messages (#5125)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.10.11..v2.11.0-preview.1

Release v2.10.11

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.21.7

Dependencies

• github.com/nats-io/nats.go v1.33.0

Improved

JetStream

• Optimize replaying messages when they are at the end of the stream (#5083) Thanks to @david-wakeo for the report!
• Optimize replaying messages with start sequence by time when there are large gaps between the time sequence and first sequence (#5088) Thanks again to @david-wakeo for the report!
• Reduce possible contention for NRG step-down (#4990)
• Reduce log-level to debug for non-actionable reallocations warnings (#5085)
• Improved placement of streams in larger clusters when created in rapid succession (#5079) Thanks to @kohlisid for the report!
• Improved filtered consumer retrieval time for large subject space streams with wildcards (#5089)

Fixed

JetStream

• Fixed a bug that could cause consumers to not match filtered subjects with wildcards correctly (#5080)
• Fixed a bug that could not allow keys/subjects to be found after restarts after extended downtime (#5054)
• Fixed a bug that would not properly update all consumer state when a filtered purge of the parent stream was narrower then the consumer’s filtered subject space (#5075)

MQTT

• Fixed an issue with retained messages not working properly if the server name had a ‘.’ (#5048)

Complete Changes

v2.10.10...v2.10.11

Release v2.10.10

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.21.6

Dependencies

• github.com/klauspost/compress v1.17.5
• github.com/nats-io/nats.go v1.32.0

Added

• Add ping_interval cluster route option to configure differently than client connections (#5029)

Improved

JetStream

• Avoid a linear scan of filestore blocks when loading messages for the first time after restart or long period of inactivity (#4969)
• NumPending calculations and subject index memory in filestore and memstore (#4960, #4983)
• Log healthy Raft group status when no longer falling behind (#4995)
• Optimize writing full filestore state where there is high subject cardinality (#5004)
• Share higher fidelity client info in JetStream advisory messages (#5019, #5026)
• More thorough gating of blocking disk I/O operations (#5022, #5027)
• Improved fallback cleanup of consumer Raft directory (#5028)

MQTT

• Use different unique names for internal consumers (#5017)

Fixed

Auth callout

• Ensure the server properly binds scoped users (#5013) Thanks to @dpotapov for the report!

Logging

• Fix missing variable in consumer skew log line (#4967)

JetStream

• Fix data race with stream stop and removal (#4963)
• Don't unconditionally send consumer snapshot when becoming leader (#4965)
• Fix tracking holes in the message block index when they are at the end of the block, improving filestore cache efficiency (#4988)
• Wait for recovery to complete before sending snapshot on stream scale-up from R1 (#5001)
• Acking a redelivered msg with more pending should move the ack floor (#5008)
• Fix for a panic calculating record length for secure erase followed by compaction (#5009)
• Reduce memory growth on interest stream recovery when consumers are ahead of the recovered stream state (#5011)
• Raft node responds to VoteRequest with outdated Term (#5021)
• Avoid sending meta leader snapshot as normal entry on leader change when not up-to-date (#5024)
• Make sure to not restart streams or consumers when they are deleted or immediately after being created (#5032)

Shutdown

• Avoid panic when reloading during lame duck mode (#5012)

Complete Changes

v2.10.9...v2.10.10

Release v2.10.9

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.21.6

Fixed

JetStream

• Guard against potential panic when finding starting sequence for stream sources from 2.9.x servers (#4950)

Complete Changes

v2.10.8...v2.10.9

Release v2.10.8

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.21.6

Dependencies

• golang.org/x/crypto v0.18.0
• golang.org/x/sys v0.16.0
• github.com/nats-io/nkeys v0.4.7

Added

TLS

• Add 'certs' option to TLS block for multi-cert support (#4889)

Improved

General

• Random number generation now uses a faster lock-free algorithm (#4901)
• no_auth_user is now allowed to be an nkey (#4938)

JetStream

• Improve matching efficiency of filter subjects in consumer (#4864) Thanks to @svenfoo for the contribution!
• Optimize JetStream metalayer snapshots by reducing allocations and simplifying marshaling (#4925)
• Micro-optimization where subject tokenization occurs (#4880) Thanks to @svenfoo for the contribution!
• Prevent backing up internal JS API requests in large-scale source and mirror setups (#4884)
• Optimize catchups for replicas and mirrors where there are a significant number of interior deletes (#4929)
• Reduce lock contention on the stream lock for some operations that could block routes & gateways (#4933)
• Do not load all blocks for NumPending when delivery is LastPerSubject (#4885)
• Call stream update only if the config has changed (#4898)
• Prevent large memory buildup in the apply queue for NRGs during startup (#4895)
• Finding the last sourced message for each source of a stream is now much faster (#4935)

MQTT

• Retained messages will now be fetched concurrently for a new subscription (#4835)

Fixed

Accounts

• Guard account random number generator with mutex (#4894) Thanks to @igorrius for the report!

JetStream

• Fix accounting for replicas and tier limits (#4868, #4909)
• Ensure all filter subjects across consumers are accounted for when purging a stream (#4873) Thanks to @svenfoo for the contribution!
• Detect corrupt psim subjects during recovery of index.db (#4890)
• Don’t allow writing snapshots to disk before recovery has completed (#4927)
• Reduce memory usage during purge operations by flushing cache (#4905)
• Return an “Account not enabled” error when trying to access JetStream via the system account (#4910)
• Reduce the number of blocks loaded into memory when doing linear scans (#4916)

Leafnodes

• Mapping updates on reload for the global account are now propagated to leafnodes correctly (#4937)
• Leafnode authorization now supports nkeys (#4940)

MQTT

• Fixed an out-of-date error message on unsupported characters in MQTT topics (#4903)

OCSP

• Default to Unknown status instead of Good for unknown status assertions (#4917)
• Fixed OCSP Stapling not resuming for gateways on reload after certs change (#4943)

Complete Changes

v2.10.7...v2.10.8

Release v2.10.7

Changelog

Refer to the 2.10 Upgrade Guide for backwards compatibility notes with 2.9.x.

Go Version

• 1.21.5

Dependencies

• github.com/klauspost/compress v1.17.4
• golang.org/x/crypto v0.16.0
• golang.org/x/sys v0.15.0
• golang.org/x/time v0.5.0

Improved

JetStream

• Increase minimum interval for full index.db state writes to reduce contention for high-speed ingest in large streams (#4858)

Fixed

JetStream

• Corruption of per-subject tracking on recover of bad or missing index.db (#4851) Thanks to @oscarwcl for the report!
• Prevent GetSeqFromTime panic in memstore if the last sequence was deleted (#4853) Thanks to @andreib1 for the report!
• Protect against bad consumer state and high-memory usage during rollback (#4857)

Complete Changes

v2.10.6...v2.10.7