amqpprox 1.1.0

amqpprox 1.1.0 🎉

amqpprox is an AMQP 0.9.1 proxy server, it is designed for use in front of an AMQP 0.9.1 compliant message queue broker such as RabbitMQ.

Rationale

Commonly people use HAProxy software acting as a load balancer to spread load between multiple machines within a serving cluster and handle failures gracefully. With amqpprox we built a similar proxy, except tailored specifically for the AMQP 0.9.1 protocol. This brings benefits which cannot be achieved with layer 4 proxying alone.

This proxy and how we use it was first publicly outlined in this talk at RabbitMQ Summit 2019, then released & introduced in detail in this talk at RabbitMQ Summit 2021.

Changelog since 1.0.0

• Support for offloading authentication requests to an http server using amqpprox_ctl AUTH SERVICE localhost 8080 /v1/amqpprox-auth?extrainfo=hardcodedextrainfo
• Change amqpprox_ctl VHOST UNPAUSE <vhost> behaviour to continue the connection if the client was paused during the handshake. This makes a pause / unpause operation (e.g. for blue-green upgrades) on average look like a single disconnection to a client.
• Implement per-vhost connection rate limits via amqpprox_ctl LIMIT CONN_RATE. An alarm-only threshold can be set via CONN_RATE_ALARM. This can be specified for an individual vhost, and/or a value for all other vhosts.
• Implement per-connection data throughput limits via amqpprox_ctl LIMIT DATA_RATE. This can be specified for an individual vhost, and/or a value for all other vhosts. An alarm only threshold can also be set.
• Added support for OpenSSL 3.0+
• Fixed some bugs:
  • Listening on multiple sockets (i.e. TLS and non-TLS in the same process) did not listen to both properly
  • Configuring two backends with the same port using ADD_DNS always failed. #63 Thanks @kriptor for the report
  • Two high-cpu usage bugs, and one correctness bug.

This project follows semver and this release should be backwards compatible with 1.0.0.

Every commit

• Add an item about streams by @alaric in #49
• README: Add reference to 2021 RabbitMQ Summit talk by @adamncasey in #51
• Add auth gate service interface by @Chinmay1412 in #11
• Fix multiple socket listeners by @adamncasey in #54
• Start running integration tests with github action CI by @Chinmay1412 in #55
• Implement Authintercept interface to communicate with configured http service by @Chinmay1412 in #53
• Start static linking protobuf library by @Chinmay1412 in #59
• Client property enrichment by @Chinmay1412 in #62
• Set custom error message inside connection close method for unauthorised client by @Chinmay1412 in #64
• BackendStore: Remove (ip,port) -> Backend mapping by @adamncasey in #65
• Add a performance testing tool by @adamncasey in #68
• Start logging correct tune method received from server by @Chinmay1412 in #71
• perf tester: Count messages/handle channel close/listen backlog by @adamncasey in #70
• Workaround asio::ssl async_read_some busy-loop by @adamncasey in #69
• Expose received broker connection close method, in case of any error, to client during handshake by @Chinmay1412 in #72
• Change from shutdown to async_shutdown and don't wait for remote by @willhoy in #73
• Add missing VHOST config command for amqpprox_ctl by @Chinmay1412 in #74
• clang-format 13 by @adamncasey in #76
• Remove unused variables by @Chinmay1412 in #77
• Disconnect or resume unpaused connections by @adamncasey in #75
• Start sending appropriate error message to the clients, if the broker mapping is not configured for particular vhost by @Chinmay1412 in #79
• Remove usage of anonymous namespace in header file by @adamncasey in #81
• boost::asio::io_service -> boost::asio::io_context by @adamncasey in #80
• Make MaybeSecureSocketAdapter::isSecure() private by @adamncasey in #78
• Change variable name to reflect instance of boost::asio::io_context class by @Chinmay1412 in #82
• Limit reason field to 255 characters in authResponse, while sending close method to amqp clients by @Chinmay1412 in #84
• Add connection rate Limiting Support in amqpprox by @Chinmay1412 in #83
• Share a bit more code between LIMIT command sections by @adamncasey in #85
• Add support to provide zero connection rate limit by @Chinmay1412 in #87
• Add Data Transfer Limits for each connection, configurable by vhost by @adamncasey in #88
• Avoid logging informational messages about the TLS handshake at ERROR by @adamncasey in #90
• Display configured data rate limits for LIMIT PRINT command by @Chinmay1412 in #89
• Add a new STAT ENABLE/DISABLE feature to skip collecting per-source data by @adamncasey in #92
• Start unit testing MaybeSecureSocketAdaptor by @adamncasey in #93
• Extend performance_tester to attempt testing many more connections by @adamncasey in #86
• Extend socket and timer lifetimes beyond data rate timer handler's by @adamncasey in #94
• Fix the CI/Conan Build System by @adamncasey in #97
• Support OpenSSL 3.0+ by @adamncasey in #98
• performance_tester: Bump cargo lock dependencies by @adamncasey in #99
• Ensure partially read frame buffer is always large enough to hold frame by @adamncasey in #101
• Reject frames larger than amqpprox max frame size by @adamncasey in #100
• add scorecard action by @willhoy in #96
• github actions: Set token permissions to read-only by @adamncasey in #102

Full Changelog: v1.0.0...v1.1.0

amqpprox 1.0.0

amqpprox 1.0.0 🎉

amqpprox is an AMQP 0.9.1 proxy server, it is designed for use in front of an AMQP 0.9.1 compliant message queue broker such as RabbitMQ.

Rationale

Commonly people use HAProxy software acting as a load balancer to spread load between multiple machines within a serving cluster and handle failures gracefully. With amqpprox we built a similar proxy, except tailored specifically for the AMQP 0.9.1 protocol. This brings benefits which cannot be achieved with layer 4 proxying alone.

This proxy and how we use it was first publicly outlined in this talk at RabbitMQ Summit 2019, then released & introduced in detail in this talk at RabbitMQ Summit 2021.

Key Advantages

• We can redirect different virtual hosts to different broker clusters
• We are able to understand/log AMQP 0.9.1 sessions passing through the proxy
• We can alter which brokers are connected to, in order to optimise network/datacenter cross-traffic.
• We can get detailed statistics without relying on the RabbitMQ broker itself
• We can easily have clients test connection failovers locally

Features

• No dependencies: config is pushed into it, does nothing by default
• Unix domain socket for control plane operations
• Allow switching connections between farms per vhost
• Load balancing between the farm members
• Resources (vhosts) can be pointed at DNS farms, not just IPs
• Statistics about each interaction
• Able to easily test client failover by severing connections on demand