Support TCP (and UDP?) in proxy rules

[a8bot]

Issue by elevran
Thursday Jun 16, 2016 at 09:01 GMT
Originally opened as https://github.com/amalgam8/sidecar/issues/4

---

Enable TCP / UDP proxy support in controller rules and sidecar

[a8bot]

Comment by rshriram
Friday Jul 15, 2016 at 20:24 GMT

---

unfortunately, openresty does not support balance_by_lua for TCP streams yet. Need to find third party modules that handle the scenario in a similar fashion.

[a8bot]

Comment by rshriram
Wednesday Aug 10, 2016 at 01:01 GMT

---

Submitted PR for balancer_by_lua support in nginx stream blocks. openresty/stream-lua-nginx-module#30

[greglanthier]

Does this PR mean there are known issues using Amalgam8 to proxy services accessed via TCP / UDP sockets like JMS servers or MongoDB instances?

[rshriram]

Hi, yes. At the moment, you would have to customize the sidecar's nginx configuration file to add the stream proxy block that allows you to proxy TCP/UDP connections. https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html

[greglanthier]

Thanks @rshriram.

Do you have a suggestion on a clean & low-touch way I could customise the Nginx configuration for the sidecar?

I have a mongo-proxy.conf file that looks like this:

 stream {
  upstream backend {
    server <ip>:27017;
  }
  server {
    listen 21212;
    proxy_connect_timeout 1s;
    proxy_timeout 1m;
    proxy_pass backend;
  }
}

that I mount into amalgam8/a8-sidecar:alpine as /etc/nginx/conf.d/mongo-proxy.conf but at startup this fails since it looks like the default nginx.conf file loads /etc/nginx/conf.d/*.conf files into the http context.

For example:

mongo_sidecar    | time="2016-11-16T21:25:46Z" level=info msg="Starting NGINX" 
mongo_sidecar    | time="2016-11-16T21:25:46Z" level=error msg="Failed to start NGINX service" error="exit status 1: nginx: [emerg] \"stream\" directive is not allowed here in /etc/nginx/conf.d/mongo_proxy.conf:1\n" 
mongo_sidecar    | time="2016-11-16T21:25:46Z" level=warning msg="Registry listener failed" error="exit status 1: nginx: [emerg] \"stream\" directive is not allowed here in /etc/nginx/conf.d/mongo_proxy.conf:1\n" 

[rshriram]

What you are doing is right. One thing that the sidecar can do is to add a stream block with an include directive that looks for files like *.stream.conf. Can you please file a bug? I will fix this in next release (which should happen in a day or so).

[greglanthier]

Thanks for your response, @rshriram.

One of the major reason I'm interested in enabling TCP & UDP sockets for a8sidecars at the moment is to take advantage of Gremlin in resiliency testing.

If the Nginx configuration issues are addressed (by extending the configuration ourselves) will that be sufficient for allowing our TCP & UDP based services to participate in Gremlin tests?

What about other a8ctl operations (route changes and so on)?

Am I interpreting things correctly if I say Lua is responsible for managing the Nginx configuration in response to A8 events?

[rshriram]

With move to Envoy, TCP proxying will be available out of the box. @ijsnellf - we just need rules support for TCP rules

[rshriram]

closing this as there is a new issue to track this for Envoy #459