VerneMQ 2.0.0

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The binary packages are compiled with OTP 26.2.4.
ARM builds in the making, Docker builds will follow.

Note on migration and incompatibilities for VerneMQ Release 2.0.0: https://docs.vernemq.com/v/master/guides/migration-to-2-0

Changelog

• New feature: Synchronized wait for retained message store
• Add option for a default rule ("*") in database Lua scripts, so that Clients can fallback to default ACLs.
• Introduce credentials obfusication that avoid printing secrets to logs and stack traces (breaking change)
• New feature: Support prometheus namespace
• New feature: Add logger support, instead of lager
• Remove deprecated subscriber format (#2247)
• Protect against empty XFF CN/Username
• Add simple options to HTTP health listener (health/ping)
• Remove deprecated allow_multiple_sessions
• Improve systemd support: Add support of systemd-notify
• New feature: Allow downgrade of client stopped due to keepalive from warning to info message (logging.keepalive_as_warning = off)
• Bugfix: Persist QoS0 to disk in case of outgoing upgrade_qos (#2220)
• 'vmq_http_api_v2': Set apikey as new default authentication method
• Bugfix: Remove 'vmq_http_pub' from default listener group and enforce apikey as default (#2222)
• New feature: "null" message store that disables persisting messages
• Add environment variable support for erlang configuration arguments
• 'vmq_admin': Introduce regex search for session show command
• 'vmq_admin': Extend vmq-admin listener show with TLS and MQTT listener settings
• Improve error reporting (include client) in logs (#2184)
• 'vmq_http_pub': Allows post in netsplit situations (follows allow_publish_during_netsplit global config)
• 'vmq_admin': Add new command tls invalide-pem-cache to support easier certificate replacement
• Add compatibility with [Erlang/OTP 26]
• Add new command to vmq-admin to clear webhook cache (webhooks cache clear)
• 'vmq_admin': Add commands allowing batch disconnects (vmq-admin session disconnect batch and vmq-admin session disconnect clients)
• 'vmq_http_pub': Allow anonymous access (allow_anonymous = on)
• New feature: Add configuration option disconnect_on_unauthorized_publish_v3 to force disconnect on unauthorized publish even for MQTT clients before v3.1.1
• New feature: Add persistent message queueing functionality to vmq_bridge Plugin, using ReplayQ

Commits

• Bump erlef/setup-beam from 1.15 to 1.16 by @dependabot in #2170
• Add Fosstodon link to README.md by @ioolkos in #2174
• Clarify v5 last will properties type by @ioolkos in #2176
• http_pub: Allow anonymous access (allow_anonymous=on) by @mths1 in #2180
• Introduces two new vmq-admin commands: disconnect clients and disconnect batch by @mths1 in #2181
• Add option to forward the Common Name (as username) of a client certi… by @ioolkos in #2183
• Bump actions/checkout from 3 to 4 by @dependabot in #2194
• Webhook plugin cache _m5 hook support. by @mths1 in #2200
• Add a new command to vmq-admin to clear webhook cache by @mths1 in #2202
• vmq_tracer: Do not crash when formatting topics without QoS on Unsubs… by @gmalkas in #2203
• vmq_tracer_cli: Read on STDIN instead of waiting for group leader exit by @gmalkas in #2204
• Apply configured max_request_line_length to HTTP requests by @ioolkos in #2208
• Initial support for OTP 26 by @mths1 in #2157
• Add CLI cmd tls invalide-pem-cache by @mths1 in #2210
• Add status command to vmq-admin node by @mths1 in #2211
• Allow plugin to post in net split situations (follows global config) by @mths1 in #2201
• Add additional logging information (print subscriber_id) by @mths1 in #2186
• Improve stability of vmq_http_pub testcase by @mths1 in #2214
• Add --tls and --mqtt to 'vmq_admin listener show' by @mths1 in #2213
• Small bugfix (empty list handling) by @mths1 in #2217
• Introduce regex search for session show (=~ syntax) and improve help message by @mths1 in #2215
• Introduce tls show command by @mths1 in #2216
• Rename listener show options show_mqtt to mqtt, and show_tls to tls by @mths1 in #2218
• Add support for environment variables for erlang configuration arguments by @mths1 in #2190
• Introduce "null" vmq_message_store_engine that disables message storage. by @mths1 in #2189
• Strengthen default config of vmq_http_pub by @mths1 in #2223
• Strengthen default config of vmq_http_v2_api by @mths1 in #2224
• Include possible scopes in vmq-admin help text by @mths1 in #2225
• Allow to persist QoS0 messages to disk in case of upgrade_qos by @mths1 in #2221
• Allow downgrade of warning "keepalive expired" from warning to info by @mths1 in #2226
• Optional feature for forced disconnect on unauthorized mqtt31 publish by @MeisterMasi in #2228
• Bump versions by @mths1 in #2227
• Update license scout script to Python3 and add a few fallbacks (redun… by @ioolkos in #2234
• Add support for systemd notify (start, watchdog) by @mths1 in #2182
• Expose more info (props) when publishing to internal plugin client by @ioolkos in #2235
• Trigger a non-zero script exit code if the cluster join command fails by @ioolkos in #2238
• Make number of generic msg store buckets configurable by @ioolkos in #2244
• Initial logger support by @mths1 in #2230
• Fix typo in ssl cli by @mths1 in #2246
• Prevent non-empty nodes from joining an SWC cluster by @ioolkos in #2239
• Remove deprecated allow_multiple_sessions by @mths1 in #2241
• Add simple options to HTTP health endpoint by @ioolkos in #2248
• Fix xff cn username by @ioolkos in #2250
• Revert "Fix xff cn username" by @ioolkos in #2252
• Remove deprecated subscriber format by @mths1 in #2247
• Initial version of vmq-web-ui by @mths1 in #2212
• Add support for prometheus namespace by @mths1 in #2255
• Add Queue behaviour definition by @ioolkos in #2245
• Disallow empty XFF CN/Username (without fallback to MQTT username) by @ioolkos in #2253
• Introduce password obfuscation by @mths1 in #2236
• Add properties to on_deliver hook, so that it can be used also for v3 clients by @mths1 in #2256
• Update to match vernemq_dev by @mths1 in #2259
• Update dependencies 2 0 by @ioolkos in #2257
• Bump erlef/setup-beam from 1.16 to 1.17 by @dependabot in #2242
• Update Copyright year by @ioolkos in #2260
• Properly initialize obfusication in test suite by @mths1 in #2261
• Fix last remaining lager log by @ioolkos in #2262
• Update default rule for all clients by @ashtonian in #2271
• Regenerate SSL test certificates by @ioolkos in #2277
• Enable the Bridge client to connect to Inet6 MQTT endpoints by @ioolkos in #2269
• Add a new delayed subscription strategy for topics with retained mess… by @mths1 in #2219

New Contributors

• @dependabot made their first contribution in #2170
• @gmalkas made their first contribution in #2203
• @MeisterMasi made their first contribution in #2228
• @ashtonian made their first contribution in #2271

Full Changelog: 1.13.0...2.0.0

VerneMQ Release Candidate 1 for 2.0.0.

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The packages are based on Erlang/OTP version OTP-26.2.2. You'll (possibly) have to re-compile your own plugins to match the OTP version.

This is a Release Candidate to allow you to test.
ARM binaries, Docker images and a migration guide for the (few) breaking changes will follow shortly.

Note on migration and incompatibilities: https://github.com/vernemq/vernemq/wiki/Migration-to-VerneMQ-2.0.0

Changelog:

• Introduce credentials obfusication that avoid printing secrets to logs and stack traces (breaking change)
• New feature: Support prometheus namespace
• New feature: Add logger support, instead of lager
• Remove deprecated subscriber format (#2247)
• Protect against empty XFF CN/Username
• Add simple options to HTTP health listener (health/ping)
• Remove deprecated allow_multiple_sessions
• Improve systemd support: Add support of systemd-notify
• New feature: Allow downgrade of client stopped due to keepalive from warning to info message (logging.keepalive_as_warning = off)
• Bugix: Persist QoS0 to disk in case of outgoing upgrade_qos (#2220)
• 'vmq_http_api_v2': Set apikey as new default authentication method
• Bugfix: Remove 'vmq_http_pub' from default listener group and enforce apikey as default (#2222)
• New feature: "null" message store that disables persisting messages
• Add environment variable support for erlang configuration arguments
• 'vmq_admin': Introduce regex search for session show command
• 'vmq_admin': Extend vmq-admin listener show with TLS and MQTT listener settings
• Improve error reporting (include client) in logs (#2184)
• 'vmq_http_pub': Allows post in netsplit situations (follows allow_publish_during_netsplit global config)
• 'vmq_admin': Add new command tls invalide-pem-cache to support easier certificate replacement
• Add compatibility with [Erlang/OTP 26]
• Add new command to vmq-admin to clear webhook cache (webhooks cache clear)
• 'vmq_admin': Add commands allowing batch disconnects (vmq-admin session disconnect batch and vmq-admin session disconnect clients)
• 'vmq_http_pub': Allow anonymous access (allow_anonymous = on)
• New feature: Add configuration option disconnect_on_unauthorized_publish_v3 to force disconnect on unauthorized publish even for MQTT clients before v3.1.1
• New feature: Add persistent message queueing functionality to vmq_bridge Plugin, using ReplayQ

VerneMQ 1.13.0

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The packages are based on Erlang/OTP version OTP-25.3.2.1. You'll (possibly) have to re-compile your own plugins to match the OTP version.

Changelog:

• New Plugin: 'vmq_http_pub', allows to ingest MQTT messages via a HTTP REST interface
• Allow configuration of max_request_line_length for HTTP(S) listeners
• Improve SWC sync handling for node leave/rejoin situations
• Improve memory footprint and performance of sessions that subscribe to many topics (new configurable vmq_reg_ordered_trie module, the old vmq_reg_trie is kept as default)
• Bugfix: Use default regview as information source for status page
• Add support for x-forward-for (XFF) header (Websockets) (#1783)
• Bugfix: QoS0 message shall ignore receive maximum setting (#2150)
• Offline queues to online queue transition can (temporarily) override the max online queue size (#1663)
• Fix processing of line endings in vmq_acl (#1897)
• QoS0 messages for offline sessions now count towards the queue_unhandeled metric (#1528,#1536)
• Allow overriding last will delay in plugins (#1998)
• Improve error logging: Report invalid modifiers in hooks
• Fix MQTT listener suspension and --kill_session flag in cluster leave.
• Adapt nodetool escript to Erlang distribution protocol at boot, enabling IPv6 compat for vmq-admi
• Expose more SWC sync protocol settings in vernemq.conf (number of SWC groups & more)
• Add vmq-admin retain delete command to CLI (single topic delete only).
• Fix per mountpoint filtering of vmq-admin retain show.
• Add 'keypasswd': Allows setting password for pem keyfile (#1676)
• Bugfix: Improve warning messages for unexpected frame type error to track origin (#1671)
• Bugfix: Remove special chars in auto-generated client id (#1673)
• Bugfix: Websocket returned error 500 and wrote to log, instead of returning 426 (protocol upgrade) #1983
• Allow to specify a maximum connection lifetime (per listener). The lifetime can be overwriten by on_register hooks.
• Improve TLSv1.3 support (Documentation, CLI, Testsuite)
• Improve HTTP/2 support for HTTPS listeners (#2117)
• Make Redis username configurable in vmq_diversity.
• Enable v5 protocol for WS and SSL listeners as a default.
• Fix dev_n builds (make dev0 dev1...).
• Fix issue #2008 where the default MQTT listener fails to create in vernemq.conf.
• Fix configuration problems when using Unix Domain Sockets.
• Add support for compilation in ARM architectures (Tested on M1 Mac and Raspberry PI). Now we can use the make rel target to build a VerneMQ release for RaspberryPI.
• CI Improvements:
  • Test on the last 3 major OTP versions, following the Erlang/OTP support conventions.
  • Test on Both Linux and OSX for each version.
  • Add more extensive smoke test by publishing/subscribing to messages.
• Add compatibility with Erlang/OTP 25.
• Allow protection of all HTTP(s) endpoints with API keys (e.g. metrics)
• Update bootstrap 4.6.2 in status page and add favicon
• vmq_passwd -c no longer overwrites existing files by default.
• Allow per-purpose HTTP endpoints (status, metrics, api) by assigning http_modules
• Add support for TLS-PSK (Pre-Shared Key) for MQTTS (TLS) listeners
• Fix regression in handling of the Proxy protocol for WebSockets.
• Refactor metrics count of active connections, using 3 new gauges: active_mqtt_connections, active_mqttws_connections (WebSocket) and total_active_connections. Adapt Status page. This also fixes an error, where the status page would show a false connection count.
• Add active_conns and all_conns info to vmq-admin listener show.

Note: tar files (like vernemq-1.13.0.alpine.x86_64.tar.gz) are the binaries used by the official Docker images. They are of no direct use.

VerneMQ 1.12.6.2

Patch Release for 1.12.6

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The packages are based on Erlang/OTP version 24.3.4.5. You'll (possibly) have to re-compile your own plugins to match the OTP version.

Changelog:

Add max_ws_frame_size setting to limit incoming WebSocket stream.

VerneMQ 1.12.6.1

Patch Release for 1.12.6

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The packages are based on Erlang/OTP version 24.3.4.5. You'll (possibly) have to re-compile your own plugins to match the OTP version.

Changelog:

• Fix bug in webhooks metrics that crashed Prometheus endpoint

VerneMQ 1.12.6

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The packages are based on Erlang/OTP version 24.3.4.5. You'll (possibly) have to re-compile your own plugins to match the OTP version.

Changelog:

• Fix outgoing QoS and outgoing QoS upgrades.
• Collect per webhook type (e.g. on_publish_m5_requests) metrics.
• Fix vmqs (SSL) inter-node communication.
• Store waiting pubrec packets in queue and add to waiting acks on reconnect.
• Fix PostGres (epgsql 4.6.0) response format in vmq_diversity.
• Fix Prometheus exposition format, related to MQTT v4 and v5 labels.
• Add Unix domain listeners (to be configured as "listener.tcp.unix_socket = local:/tmp/my.sock:0")
• Add new setting tls_handshake_timeout option to SSL listeners.
• Fix webhook cache to not allow multiple entries per key
• Fix recovery of waiting pubrecs in MQTT v5
• Update epgsql to 4.6.1
• Update eredis to 1.7.0 (vmq_diversity)
• Update eredis to 1.7.0 (vmq_diversity)
• Update erlang-mongodb to ref 713e8bd (vmq_diversity)
• Update sext to 1.8.0
• Update stdout_formatter to 0.2.4

VerneMQ Release 1.12.5

Changelog:

• Change git clone protocol to https globally.
• Update erlang-pbkdf2 and mongodb-erlang, fixes Issue #1928.
• Fix proxy_protocol_use_cn_as_username config for WS listeners: the CN now overrides the MQTT username correctly.
• Add ?P_RESPONSE_INFO property in CONNACK when auth_on_register_m5_hook sets the property.
• Add per-listener allow_anonymous_override config (to open a single listener while allow_anonymous=off).
• Update certifi to 2.9.0.
• Update epgsql to 4.6.0.
• Update jsx to 3.1.0 (vmq_diversity, vmq_webhooks).
• Update eredis to 1.4.1 (vmq_diversity).

VerneMQ Release 1.12.4

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The packages are based on Erlang/OTP version 24.2 You'll (possibly) have to re-compile your own plugins to match the OTP version.

Changelog:

• Add additional bcrypt configuration options
• Bump bcrypt version
• Switch on V5 in MQTT listeners as a default
• Fix v5 username regression, interfering with auth_on_register modifiers.
• Move the add_session on queue draining from error to info logging.
• Add vmq-admin trace stop_all command to stop unreachable traces.
• Fix bridge configuration regarding CAP settings.
• Add prefer_online_before_local shared subscription policy.

Technical note:
Some of the DB drivers in vmq_diversity showed unexpected issues with OTP 24. We're fixing those in main branch & for the next release. If you need MongoDB or PostGres auth plugins, you can use release 1.12.3.

VerneMQ Release 1.12.3

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The packages are based on Erlang/OTP version 23.3.2. You'll (possibly) have to re-compile your own plugins to match the OTP version.

Changelog: (Quickfix Release to 1.12.2, to fix SWC sync regression)

• Move from r_mode to read preference in VMQ Diversity MongoDB
• Fix SWC event broadcast regression, which led to higher SWC sync times.

VerneMQ Release 1.12.2

Note:

VerneMQ binary software distribution packages and Docker images are covered by the VerneMQ EULA. See the blog post for more information:
https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html.

TL;DR: You are required to contribute a yearly usage fee (subscription), in case you use these packages commercially.

The packages are based on Erlang/OTP version 23.3.2. You'll (possibly) have to re-compile your own plugins to match the OTP version.

Changelog:

• Fix username for allow_anonymous case in MQTT v5 (keep the given username in session)
• Fix Retain Server cache race condition for messages originating from local node.
• Remove debug logging statement for MongoDB in vmq_diversity
• Allow Overlay_vars from include files in vars.generated
• Switch to no_dot_erlang instead of start_clean in Runner script.