Releases: emqx/emqx
EMQX v5.6.1
Bug Fixes
-
#12759 EMQX now automatically removes invalid backup files that fail during upload due to schema validation errors. This fix ensures that only valid configuration files are displayed and stored, enhancing system reliability.
-
#12766 Renamed
message_queue_too_long
error reason tomailbox_overflow
mailbox_overflow
. The latter is consistent with the corresponding config parameter:force_shutdown.max_mailbox_size
. -
#12773 Upgraded HTTP client libraries.
The HTTP client library (
gun-1.3
) incorrectly appended a:portnumber
suffix to theHost
header for
standard ports (http
on port 80,https
on port 443). This could cause compatibility issues with servers or gateways performing strictHost
header checks (e.g., AWS Lambda, Alibaba Cloud HTTP gateways), leading to errors such asInvalidCustomDomain.NotFound
or "The specified CustomDomain does not exist." -
#12802 Improved how EMQX handles node removal from clusters via the
emqx ctl cluster leave
command. Previously, nodes could unintentionally rejoin the same cluster (unless it was stopped) if the configured clusterdiscovery_strategy
was notmanual
. With the latest update, executing thecluster leave
command now automatically disables cluster discovery for the node, preventing it from rejoining. To re-enable cluster discovery, use theemqx ctl discovery enable
command or simply restart the node. -
#12814 Improved error handling for the
/clients/{clientid}/mqueue_messages
and/clients/{clientid}/inflight_messages
APIs in EMQX. These updates address:- Internal Timeout: If EMQX fails to retrieve the list of Inflight or Mqueue messages within the default 5-second timeout, likely under heavy system load, the API will return 500 error with the response
{"code":"INTERNAL_ERROR","message":"timeout"}
, and log additional details for troubleshooting. - Client Shutdown: Should the client connection be terminated during an API call, the API now returns a 404 error, with the response
{"code": "CLIENT_SHUTDOWN", "message": "Client connection has been shutdown"}
. This ensures clearer feedback when client connections are interrupted.
- Internal Timeout: If EMQX fails to retrieve the list of Inflight or Mqueue messages within the default 5-second timeout, likely under heavy system load, the API will return 500 error with the response
-
#12824 Updated the statistics metrics
subscribers.count
andsubscribers.max
to include shared subscribers. Previously, these metrics accounted only for non-shared subscribers. -
#12826 Fixed issues related to the import functionality of source data integrations and retained messages in EMQX. Before this update:
- The data integration sources specified in backup files were not being imported. This included configurations under the
sources.mqtt
category with specific connectors and parameters such as QoS and topics. - Importing the
mnesia
table for retained messages was not supported.
- The data integration sources specified in backup files were not being imported. This included configurations under the
-
#12843 Fixed
cluster_rpc_commit
transaction ID cleanup procedure on replicator nodes after executing theemqx ctl cluster leave
command. Previously, failing to properly clear these transaction IDs impeded configuration updates on the core node. -
#12885 Fixed an issue in EMQX where users were unable to view "Retained Messages" under the "Monitoring" menu in the Dashboard.
The "Retained messages" backend API uses the
qlc
library. This problem was due to a permission issue where theqlc
library'sfile_sorter
function tried to use a non-writable directory,/opt/emqx
, to store temporary files, resulting from recent changes in directory ownership permissions in Docker deployments.This update modifies the ownership settings of the
/opt/emqx
directory toemqx:emqx
, ensuring that all necessary operations, including retained messages retrieval, can proceed without access errors.
EMQX Enterprise 5.6.1
Bug Fixes
-
#12759 EMQX now automatically removes invalid backup files that fail during upload due to schema validation errors. This fix ensures that only valid configuration files are displayed and stored, enhancing system reliability.
-
#12766 Renamed
message_queue_too_long
error reason tomailbox_overflow
mailbox_overflow
. The latter is consistent with the corresponding config parameter:force_shutdown.max_mailbox_size
. -
#12773 Upgraded HTTP client libraries.
The HTTP client library (
gun-1.3
) incorrectly appended a:portnumber
suffix to theHost
header for
standard ports (http
on port 80,https
on port 443). This could cause compatibility issues with servers or gateways performing strictHost
header checks (e.g., AWS Lambda, Alibaba Cloud HTTP gateways), leading to errors such asInvalidCustomDomain.NotFound
or "The specified CustomDomain does not exist." -
#12802 Improved how EMQX handles node removal from clusters via the
emqx ctl cluster leave
command. Previously, nodes could unintentionally rejoin the same cluster (unless it was stopped) if the configured clusterdiscovery_strategy
was notmanual
. With the latest update, executing thecluster leave
command now automatically disables cluster discovery for the node, preventing it from rejoining. To re-enable cluster discovery, use theemqx ctl discovery enable
command or simply restart the node. -
#12814 Improved error handling for the
/clients/{clientid}/mqueue_messages
and/clients/{clientid}/inflight_messages
APIs in EMQX. These updates address:- Internal Timeout: If EMQX fails to retrieve the list of Inflight or Mqueue messages within the default 5-second timeout, likely under heavy system load, the API will return 500 error with the response
{"code":"INTERNAL_ERROR","message":"timeout"}
, and log additional details for troubleshooting. - Client Shutdown: Should the client connection be terminated during an API call, the API now returns a 404 error, with the response
{"code": "CLIENT_SHUTDOWN", "message": "Client connection has been shutdown"}
. This ensures clearer feedback when client connections are interrupted.
- Internal Timeout: If EMQX fails to retrieve the list of Inflight or Mqueue messages within the default 5-second timeout, likely under heavy system load, the API will return 500 error with the response
-
#12824 Updated the statistics metrics
subscribers.count
andsubscribers.max
to include shared subscribers. Previously, these metrics accounted only for non-shared subscribers. -
#12826 Fixed issues related to the import functionality of source data integrations and retained messages in EMQX. Before this update:
- The data integration sources specified in backup files were not being imported. This included configurations under the
sources.mqtt
category with specific connectors and parameters such as QoS and topics. - Importing the
mnesia
table for retained messages was not supported.
- The data integration sources specified in backup files were not being imported. This included configurations under the
-
#12843 Fixed
cluster_rpc_commit
transaction ID cleanup procedure on replicator nodes after executing theemqx ctl cluster leave
command. Previously, failing to properly clear these transaction IDs impeded configuration updates on the core node. -
#12882 Fixed an issue with the RocketMQ action in EMQX data integration, ensuring that messages are correctly routed to their configured topics. Previously, when multiple actions shared a single RocketMQ connector, all messages were mistakenly sent to the topic configured for the first action. This fix involves starting a distinct set of RocketMQ workers for each topic, preventing cross-topic message delivery errors.
-
#12885 Fixed an issue in EMQX where users were unable to view "Retained Messages" under the "Monitoring" menu in the Dashboard.
The "Retained messages" backend API uses the
qlc
library. This problem was due to a permission issue where theqlc
library'sfile_sorter
function tried to use a non-writable directory,/opt/emqx
, to store temporary files, resulting from recent changes in directory ownership permissions in Docker deployments.This update modifies the ownership settings of the
/opt/emqx
directory toemqx:emqx
, ensuring that all necessary operations, including retained messages retrieval, can proceed without access errors.
EMQX v5.6.0
Enhancements
-
#12251 Optimized the performance of the RocksDB-based persistent sessions, achieving a reduction in RAM usage and database request frequency. Key improvements include:
- Introduced dirty session state to avoid frequent mria transactions.
- Introduced an intermediate buffer for the persistent messages.
- Used separate tracks of PacketIds for QoS1 and QoS2 messages.
- Limited the number of continuous ranges of inflight messages to 1 per stream.
-
#12326 Enhanced session tracking with registration history. EMQX now has the capability to monitor the history of session registrations, including those that have expired. By configuring
broker.session_history_retain
, EMQX retains records of expired sessions for a specified duration.-
Session count API: Use the API
GET /api/v5/sessions_count?since=1705682238
to obtain a count of sessions across the cluster that remained active since the given UNIX epoch timestamp (with seconds precision). This enhancement aids in analyzing session activity over time. -
Metrics expansion with cluster sessions gauge: A new gauge metric,
cluster_sessions
, is added to better track the number of sessions within the cluster. This metric is also integrated into Prometheus for easy monitoring:# TYPE emqx_cluster_sessions_count gauge emqx_cluster_sessions_count 1234
NOTE: Please consider this metric as an approximate estimation. Due to the asynchronous nature of data collection and calculation, exact precision may vary.
-
-
#12338 Introduced a time-based garbage collection mechanism to the RocksDB-based persistent session backend. This feature ensures more efficient management of stored messages, optimizing storage utilization and system performance by automatically purging outdated messages.
-
#12398 Exposed the
swagger_support
option in the Dashboard configuration, allowing for the enabling or disabling of the Swagger API documentation. -
#12467 Started supporting cluster discovery using AAAA DNS record type.
-
#12483 Renamed
emqx ctl conf cluster_sync tnxid ID
toemqx ctl conf cluster_sync inspect ID
.For backward compatibility,
tnxid
is kept, but considered deprecated and will be removed in 5.7. -
#12499 Enhanced client banning capabilities with extended rules, including:
- Matching
clientid
against a specified regular expression. - Matching client's
username
against a specified regular expression. - Matching client's peer address against a CIDR range.
Important Notice: Implementing a large number of broad matching rules (not specific to an individual clientid, username, or host) may affect system performance. It's advised to use these extended ban rules judiciously to maintain optimal system efficiency.
- Matching
-
#12509 Implemented API to re-order all authenticators / authorization sources.
-
#12517 Configuration files have been upgraded to accommodate multi-line string values, preserving indentation for enhanced readability and maintainability. This improvement utilizes
"""~
and~"""
markers to quote indented lines, offering a structured and clear way to define complex configurations. For example:rule_xlu4 { sql = """~ SELECT * FROM "t/#" ~""" }
See HOCON 0.42.0 release notes for details.
-
#12520 Implemented log throttling. The feature reduces the volume of logged events that could potentially flood the system by dropping all but the first occurance of an event within a configured time window.
Log throttling is applied to the following log events that are critical yet prone to repetition:authentication_failure
authorization_permission_denied
cannot_publish_to_topic_due_to_not_authorized
cannot_publish_to_topic_due_to_quota_exceeded
connection_rejected_due_to_license_limit_reached
dropped_msg_due_to_mqueue_is_full
-
#12561 Implemented HTTP APIs to get the list of client's in-flight and message queue (mqueue) messages. These APIs facilitate detailed insights and effective control over message queues and in-flight messaging, ensuring efficient message handling and monitoring.
To get the first chunk of data:
GET /clients/{clientid}/mqueue_messages?limit=100
GET /clients/{clientid}/inflight_messages?limit=100
Alternatively, for the first chunks without specifying a start position:
GET /clients/{clientid}/mqueue_messages?limit=100&position=none
GET /clients/{clientid}/inflight_messages?limit=100&position=none
To get the next chunk of data:
GET /clients/{clientid}/mqueue_messages?limit=100&position={position}
GET /clients/{clientid}/inflight_messages?limit=100&position={position}
Where
{position}
is a value (opaque string token) ofmeta.position
field from the previous response.Ordering and Prioritization:
- Mqueue Messages: These are prioritized and sequenced based on their queue order (FIFO), from higher to lower priority. By default, mqueue messages carry a uniform priority level of 0.
- In-Flight Messages: Sequenced by the timestamp of their insertion into the in-flight storage, from oldest to newest.
-
#12590 Removed
mfa
meta data from log messages to improve clarity. -
#12641 Improved text log formatter fields order. The new fields order is as follows:
tag
>clientid
>msg
>peername
>username
>topic
> [other fields] -
#12670 Added field
shared_subscriptions
to endpoint/monitor_current
and/monitor_current/nodes/:node
. -
#12679 Upgraded docker image base from Debian 11 to Debian 12.
-
#12700 Started supporting "b" and "B" unit in bytesize hocon fields. For example, all three fields below will have the value of 1024 bytes:
bytesize_field = "1024b" bytesize_field2 = "1024B" bytesize_field2 = 1024
-
#12719 The
/clients
API has been upgraded to accommodate queries for multipleclientid
s andusername
s simultaneously, offering a more flexible and powerful tool for monitoring client connections. Additionally, this update introduces the capability to customize which client information fields are included in the API response, optimizing for specific monitoring needs.Examples of Multi-Client/Username Queries:
- To query multiple clients by ID:
/clients?clientid=client1&clientid=client2
- To query multiple users:
/clients?username=user11&username=user2
- To combine multiple client IDs and usernames in one query:
/clients?clientid=client1&clientid=client2&username=user1&username=user2
Examples of Selecting Fields for the Response:
- To include all fields in the response:
/clients?fields=all
(Note: Omitting thefields
parameter defaults to returning all fields.) - To specify only certain fields:
/clients?fields=clientid,username
- To query multiple clients by ID:
-
#12381 Added new SQL functions:
map_keys()
,map_values()
,map_to_entries()
,join_to_string()
,join_to_string()
,join_to_sql_values_string()
,is_null_var()
,is_not_null_var()
.For more information on the functions and their usage, refer to Built-in SQL Functions the documentation.
-
#12336 Performance enhancement. Created a dedicated async task handler pool to handle client session cleanup tasks.
-
#12725 Implemented REST API to list the available source types.
-
#12746 Added
username
log field. If MQTT client is connected with a non-empty username the logs and traces will includeusername
field. -
#12785 Added
timestamp_format
configuration option to log handlers. This new option allows for the following settings:-
auto
: Automatically determines the timestamp format based on the log formatter being used.
Utilizesrfc3339
format for text formatters, andepoch
format for JSON formatters. -
epoch
: Represents timestamps in microseconds precision Unix epoch format. -
rfc3339
: Uses RFC3339 compliant format for date-time strings. For example,2024-03-26T11:52:19.777087+00:00
.
-
Bug Fixes
-
#11868 Fixed a bug where will messages were not published after session takeover.
-
#12347 Implemented an update to ensure that messages processed by the Rule SQL for the MQTT egress data bridge are always rendered as valid, even in scenarios where the data is incomplete or lacks certain placeholders defined in the bridge configuration. This adjustment prevents messages from being incorrectly deemed invalid and subsequently discarded by the MQTT egress data bridge, as was the case previously.
When variables in
payload
andtopic
templates are undefined, they are now rendered as empty strings instead of the literalundefined
string. -
#12472 Fixed an issue whe...
EMQX Enterprise 5.6.0
Enhancements
-
#12326 Enhanced session tracking with registration history. EMQX now has the capability to monitor the history of session registrations, including those that have expired. By configuring
broker.session_history_retain
, EMQX retains records of expired sessions for a specified duration.-
Session count API: Use the API
GET /api/v5/sessions_count?since=1705682238
to obtain a count of sessions across the cluster that remained active since the given UNIX epoch timestamp (with seconds precision). This enhancement aids in analyzing session activity over time. -
Metrics expansion with cluster sessions gauge: A new gauge metric,
cluster_sessions
, is added to better track the number of sessions within the cluster. This metric is also integrated into Prometheus for easy monitoring:# TYPE emqx_cluster_sessions_count gauge emqx_cluster_sessions_count 1234
NOTE: Please consider this metric as an approximate estimation. Due to the asynchronous nature of data collection and calculation, exact precision may vary.
-
-
#12398 Exposed the
swagger_support
option in the Dashboard configuration, allowing for the enabling or disabling of the Swagger API documentation. -
#12467 Started supporting cluster discovery using AAAA DNS record type.
-
#12483 Renamed
emqx ctl conf cluster_sync tnxid ID
toemqx ctl conf cluster_sync inspect ID
.For backward compatibility,
tnxid
is kept, but considered deprecated and will be removed in 5.7. -
#12495 Introduced new AWS S3 connector and action.
-
#12499 Enhanced client banning capabilities with extended rules, including:
- Matching
clientid
against a specified regular expression. - Matching client's
username
against a specified regular expression. - Matching client's peer address against a CIDR range.
Important Notice: Implementing a large number of broad matching rules (not specific to an individual clientid, username, or host) may affect system performance. It's advised to use these extended ban rules judiciously to maintain optimal system efficiency.
- Matching
-
#12509 Implemented API to re-order all authenticators / authorization sources.
-
#12517 Configuration files have been upgraded to accommodate multi-line string values, preserving indentation for enhanced readability and maintainability. This improvement utilizes
"""~
and~"""
markers to quote indented lines, offering a structured and clear way to define complex configurations. For example:rule_xlu4 { sql = """~ SELECT * FROM "t/#" ~""" }
See HOCON 0.42.0 release notes for details.
-
#12520 Implemented log throttling. The feature reduces the volume of logged events that could potentially flood the system by dropping all but the first occurance of an event within a configured time window.
Log throttling is applied to the following log events that are critical yet prone to repetition:authentication_failure
authorization_permission_denied
cannot_publish_to_topic_due_to_not_authorized
cannot_publish_to_topic_due_to_quota_exceeded
connection_rejected_due_to_license_limit_reached
dropped_msg_due_to_mqueue_is_full
-
#12561 Implemented HTTP APIs to get the list of client's in-flight and message queue (mqueue) messages. These APIs facilitate detailed insights and effective control over message queues and in-flight messaging, ensuring efficient message handling and monitoring.
To get the first chunk of data:
GET /clients/{clientid}/mqueue_messages?limit=100
GET /clients/{clientid}/inflight_messages?limit=100
Alternatively, for the first chunks without specifying a start position:
GET /clients/{clientid}/mqueue_messages?limit=100&position=none
GET /clients/{clientid}/inflight_messages?limit=100&position=none
To get the next chunk of data:
GET /clients/{clientid}/mqueue_messages?limit=100&position={position}
GET /clients/{clientid}/inflight_messages?limit=100&position={position}
Where
{position}
is a value (opaque string token) ofmeta.position
field from the previous response.Ordering and Prioritization:
- Mqueue Messages: These are prioritized and sequenced based on their queue order (FIFO), from higher to lower priority. By default, mqueue messages carry a uniform priority level of 0.
- In-Flight Messages: Sequenced by the timestamp of their insertion into the in-flight storage, from oldest to newest.
-
#12590 Removed
mfa
meta data from log messages to improve clarity. -
#12641 Improved text log formatter fields order. The new fields order is as follows:
tag
>clientid
>msg
>peername
>username
>topic
> [other fields] -
#12670 Added field
shared_subscriptions
to endpoint/monitor_current
and/monitor_current/nodes/:node
. -
#12679 Upgraded docker image base from Debian 11 to Debian 12.
-
#12700 Started supporting "b" and "B" unit in bytesize hocon fields.
For example, all three fields below will have the value of 1024 bytes:
bytesize_field = "1024b" bytesize_field2 = "1024B" bytesize_field2 = 1024
-
#12719 The
/clients
API has been upgraded to accommodate queries for multipleclientid
s andusername
s simultaneously, offering a more flexible and powerful tool for monitoring client connections. Additionally, this update introduces the capability to customize which client information fields are included in the API response, optimizing for specific monitoring needs.Examples of Multi-Client/Username Queries:
- To query multiple clients by ID:
/clients?clientid=client1&clientid=client2
- To query multiple users:
/clients?username=user11&username=user2
- To combine multiple client IDs and usernames in one query:
/clients?clientid=client1&clientid=client2&username=user1&username=user2
Examples of Selecting Fields for the Response:
- To include all fields in the response:
/clients?fields=all
(Note: Omitting thefields
parameter defaults to returning all fields.) - To specify only certain fields:
/clients?fields=clientid,username
- To query multiple clients by ID:
-
#12330 The Cassandra bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12353 The OpenTSDB bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12376 The Kinesis bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12386 The GreptimeDB bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12423 The RabbitMQ bridge has been split into connector, action and source components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12425 The ClickHouse bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12439 The Oracle bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12449 The TDEngine bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12488 The RocketMQ bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12512 The HStreamDB bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically, however, it is recommended to do the upgrade manually as new fields have been added to the configuration.
-
#12543 The DynamoDB bridge has been split into connector and action components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12595 The Kafka Consumer bridge has been split into connector and source components. They are backwards compatible with the bridge HTTP API. Configuration will be upgraded automatically.
-
#12619 The Microsoft SQL Server bridge has been split into connector...
EMQX v5.5.1
5.5.1
Bug Fixes
-
#12471 Fixed an issue that data integration configurations failed to load correctly during upgrades from EMQX version 5.0.2 to newer releases.
-
#12598 Fixed an issue that users were unable to subscribe to or unsubscribe from shared topic filters via HTTP API.
The affected APIs include:
-
/clients/:clientid/subscribe
-
/clients/:clientid/subscribe/bulk
-
/clients/:clientid/unsubscribe
-
/clients/:clientid/unsubscribe/bulk
-
-
#12601 Fixed an issue where logs of the LDAP driver were not being captured. Now, all logs are recorded at the
info
level. -
#12606 The Prometheus API experienced crashes when the specified SSL certificate file did not exist in the given path. Now, when an SSL certificate file is missing, the
emqx_cert_expiry_at
metric will report a value of 0, indicating the non-existence of the certificate. -
#12620 Redacted sensitive information in HTTP headers to exclude authentication and authorization credentials from
debug
level logs in the HTTP Server connector, mitigating potential security risks. -
#12632 Fixed an issue where the rule engine's SQL built-in function
date_to_unix_ts
produced incorrect timestamp results for dates starting from March 1st on leap years.
EMQX Enterprise e5.5.1
Enhancements
- #12497 Improved MongoDB connector performance, resulting in more efficient database interactions. This enhancement is supported by improvements in the MongoDB Erlang driver as well (mongodb-erlang PR).
Bug Fixes
-
#12471 Fixed an issue that data integration configurations failed to load correctly during upgrades from EMQX version 5.0.2 to newer releases.
-
#12598 Fixed an issue that users were unable to subscribe to or unsubscribe from shared topic filters via HTTP API.
The affected APIs include:
-
/clients/:clientid/subscribe
-
/clients/:clientid/subscribe/bulk
-
/clients/:clientid/unsubscribe
-
/clients/:clientid/unsubscribe/bulk
-
-
#12601 Fixed an issue where logs of the LDAP driver were not being captured. Now, all logs are recorded at the
info
level. -
#12606 The Prometheus API experienced crashes when the specified SSL certificate file did not exist in the given path. Now, when an SSL certificate file is missing, the
emqx_cert_expiry_at
metric will report a value of 0, indicating the non-existence of the certificate. -
#12608 Fixed a
function_clause
error in the IoTDB action caused by the absence of apayload
field in query data. -
#12610 Fixed an issue where connections to the LDAP connector could unexpectedly disconnect after a certain period of time.
-
#12620 Redacted sensitive information in HTTP headers to exclude authentication and authorization credentials from
debug
level logs in the HTTP Server connector, mitigating potential security risks. -
#12632 Fixed an issue where the rule engine's SQL built-in function
date_to_unix_ts
produced incorrect results for dates starting from March 1st on leap years.
EMQX Enterprise 5.6.0-alpha.2
Merge pull request #12644 from thalesmg/prepare-560-alpha1-r56-20240304 fix docker repository enumeration and prepare `e5.6.0-alpha.2`
EMQX v5.5.1-rc.1
chore: bump version to 5.5.1-rc.1
EMQX Enterprise 5.5.1-rc.1
e5.5.1-rc.1 chore: bump version to 5.5.1-rc.1
EMQX v5.5.0
v5.5.0
Enhancements
-
#12085 EMQX has been upgraded to leverage the capabilities of OTP version 26.1.2-2. NOTE: Docker images are still built with OTP 25.3.2.
-
#12189 Enhanced the ACL claim format in EMQX JWT authentication for greater versatility. The updated format now supports an array structure, aligning more closely with the file-based ACL rules.
For example:
[ { "permission": "allow", "action": "pub", "topic": "${username}/#", "qos": [0, 1], "retain": true }, { "permission": "allow", "action": "sub", "topic": "eq ${username}/#", "qos": [0, 1] }, { "permission": "deny", "action": "all", "topics": ["#"] } ]
In this new format, the absence of a matching rule does not result in an automatic denial of the action. The authorization chain can advance to other configured authorizers if a match is not found in the JWT ACL. If no match is found throughout the chain, the final decision defers to the default permission set in
authorization.no_match
. -
#12267 Added a new
timeout
parameter to thecluster/:node/invite
interface, addressing the issue of default timeouts.
The previously set 5-second default timeout often led to HTTP API call timeouts because joining an EMQX cluster usually requires more time.In addition, EMQX added a new API
/cluster/:node/invite_async
to support an asynchronous way to invite nodes to join the cluster and introduced a newcluster/invitation
API to inspect the join status. -
#12272 Introduced updates to the
retain
API in EMQX:- Added a new API
DELETE /retainer/messages
to clean all retained messages. - Added an optional topic filter parameter
topic
in the query string for the APIGET /retainer/messages
. For example, using a query stringtopic=t/1
filters the retained messages for a specific topic, improving the efficiency of message retrieval.
- Added a new API
-
#12277 Added
mqtt/delayed/messages/:topic
API to remove delayed messages by topic name. -
#12278 Adjusted the maximum pagination size for paginated APIs in the REST API from
3000
to10000
. -
#12289 Authorization caching now supports the exclusion of specific topics. For the specified list of topics and topic filters, EMQX will not generate an authorization cache. The list can be set through the
authorization.cache.excludes
configuration item or via the Dashboard. For these specific topics, permission checks will always be conducted in real-time rather than relying on previous cache results, thus ensuring the timeliness of authorization outcomes. -
#12329 Added
broker.routing.batch_sync
configuration item to enable a dedicated process pool that synchronizes subscriptions with the global routing table in batches, thus reducing the frequency of cross-node communication that can be slowed down by network latency. Processing multiple subscription updates collectively, not only accelerates synchronization between replica nodes and core nodes in a cluster but also reduces the load on the broker pool, minimizing the risk of overloading. -
#12333 Added a
tags
field for actions and connectors. Similar to thedescription
field (which is a free text annotation),tags
can be used to annotate actions and connectors for filtering and grouping. -
#12299 Exposed more metrics to improve observability:
Montior API:
- Added
retained_msg_count
field to/api/v5/monitor_current
. - Added
license_quota
field to/api/v5/monitor_current
- Added
retained_msg_count
andnode_uptime
fields to/api/v5/monitor_current/nodes/{node}
. - Added
retained_msg_count
,license_quota
andnode_uptime
fields to/api/v5/monitor_current/nodes/{node}
.
Prometheus API:
- Added
emqx_cert_expiry_at
andemqx_license_expiry_at
to/api/v5/prometheus/stats
to display TLS listener certificate expiration time and license expiration time. - Added
/api/v5/prometheus/auth
endpoint to provide metrics such as execution count and running status for all authenticatiors and authorizators. - Added
/api/v5/prometheus/data_integration
endpoint to provide metrics such as execution count and status for all rules, actions, and connectors.
Limitations:
Prometheus push gateway only supports the content in/api/v5/prometheus/stats?mode=node
.For more API details and metric type information, please see swagger api docs.
- Added
-
#12196 Improved network efficiency during routes cleanup.
Previously, when a node was down, a delete operation for each route to that node must be exchanged between all the other live nodes. After this change, only one
match and delete
operation is exchanged between all live nodes, significantly reducing the number of necessary network packets and decreasing the load on the inter-cluster network.
This optimization must be especially helpful for geo-distributed EMQX deployments where network latency can be significantly high. -
#12354 The concurrent creation and updates of data integrations are now supported, significantly increasing operation speeds, such as when importing backup files.
Bug Fixes
-
#12232 Fixed an issue when cluster commit log table was not deleted after a node was forced to leave a cluster.
-
#12243 Fixed a family of subtle race conditions that could lead to inconsistencies in the global routing state.
-
#12269 Improved error handling in the
/clients
interface; now returns a 400 status with more detailed error messages, instead of a generic 500, for query string validation failures. -
#12285 Updated the CoAP gateway to support short parameter names for slight savings in datagram size. For example,
clientid=bar
can be written asc=bar
. -
#12303 Fixed the message indexing in retainer. Previously, clients with wildcard subscriptions might receive irrelevant retained messages not matching their subscription topics.
-
#12305 Corrected an issue with incomplete client/connection information being passed into
emqx_cm
, which could lead to internal inconsistencies and affect memory usage and operations like node evacuation. -
#12306 Fixed an issue preventing the connectivity test for the Connector from functioning correctly after updating the password parameter via the HTTP API.
-
#12359 Fixed an issue causing error messages when restarting a node configured with some types of data bridges. Additionally, these bridges were at risk of entering a failed state upon node restart, requiring a manual restart to restore functionality.
-
#12404 Fixed an issue where restarting a data integration with heavy message flow could lead to a stop in the collection of data integration metrics.