File Based Configuration

[dgduncan]

Wanted to start a new thread as to not clutter up the roadmap discussion.
I have implemented a completely barebones implementation of file based config using a yml file and a flag passed in when running main.go here. I have some questions that I would love to get everyones opinion on so I know where to go from here:

Architecture:

• Create separate mqtt.Options struct or simply tag the current struct?
• Should this live in it's own new package, say config or configuration where a user of the library can choose to use the functionality from the package or should it be integrated into the server directly.
• Should I update the Dockerfile to include a env/flag so when running the default Dockerfile a user can simply include a filepath or filename?

Features:
I want to know what everyone thinks is mandatory to make this useful, and a nice to have. Here are my opinions but I want to know more.

Mandatory:

• Server configuration (i.e a file copy of what Options looks like when creating a server`)
• Ports
• Log level

Useful:

• SSL with basic certificate functionality (i.e file/filepath)
• Basic Hooks

Nice to have

• Non standard hooks

Please feel free mention anything else and give ideas!

Hooks

• AllowAll
• Access Ledger
• Bolt
• Badger
• Redis
• Debug

Listeners

• Healthcheck
• Stats
• TCP
• Websocket
• UnixSock
• Net
• TLS

Servers

• Capabilities

[werbenhu]

I suggest that the relevant configurations inside DefaultServerCapabilities can be written to the configuration file, such as MaximumClientWritesPending. When the number of connections reaches a certain level, these parameters have a significant impact on memory. By placing them in the configuration file, users can fine-tune these parameters according to their specific scenarios.

[dgduncan]

I have added a working version, albeit not a feature complete implementation, of file based configuration here. here are some of the key choices I made and would love any considerations.

• Static config file name mochi_config.yml
• New package called config where Configure returns a *mqtt.Server instance. Will most likely create a new Config interface to allow new Configs to be created say from std.in, remote server, etc
• Server capabilities is the same struct used by the server but with yaml tags that have been changed to snake case to fit with the more common, but not standard, yaml styling
• New "smart" defaults created such that if a user only adds a few config fields to the mochi_config.yml file, defaults are created for the other fields
• default mochi docker should look for mochi_config.yml file but will not require it. This should lower barrier of entry to using the docker file as I have found config files can be confusing to new users compared to allowing to run with a very basic default to get a user up and running
• Config file is accessible via mounting a file volume when starting the image.

here is an example of the docker command:
docker run -p 1883:1883 -p 8081:8081 -v ${PWD}/mochi_config.yml:/mochi_config.yml test_docker:v0.0.1

[SimonMacIntyre]

That log file was only missing because I think I caused the docker image to explode (quietly) when it tried to bind to a port that I did not have available. That's my theory.. see my edits above with the strikeout!

One thing to note that isn't shown in that above file link is that for the TLS config there is indeed a config for the cacert in the form of "cacert_file"

Hmm how do i test it? In your example I see this part of the yaml:

    tls:
      certfile: cert.crt
      keyfile: key.txt

Are you saying that I should call it, cacert_file instead?

I tried this:

    tls:
     certfile: /etc/ssl/certs/cert.pem
     keyfile: /etc/ssl/certs/cert.pem

The server started, but I was not able to load my https 8080 dashboard. Also in the logs it said ws not wss, and http not https for the listeners.

So I am probably misconfiguring something wrong here 🤔

Here's what I'm using to test:

server:
  hooks:
    allow_all: true
  listeners:
    healthcheck:
      port: 8081
    stats:
      port: 8080
    tcp:
      port: 8883
    websocket:
      port: 8882
    tls:
      certfile: /etc/ssl/certs/cert.pem
      keyfile: /etc/ssl/certs/cert.pem
  options:
    capabilities:
      maximum_message_expiry_interval: 100
      compatibilities:
        obscure_not_authorized: true

and docker compose:

  mqtt2:
    image: dgduncan93/mochi-server-file-config:v0.0.1
    ports:
      - 8883:8883
      - 8882:8882
      - 8080:8080
    environment:
      VIRTUAL_HOST: mqtt.xrdm.docker
    networks:
      - xrdm-local
    volumes:
      - ~/.arborxr/certs/xrdm.docker.crt:/etc/ssl/certs/cert.pem:ro
      - ~/.arborxr/certs/xrdm.docker.key:/etc/ssl/certs/key.pem:ro
      - ./mochi_config.yml:/mochi_config.yml

One last thing, I noticed you mentioned:

New "smart" defaults created such that if a user only adds a few config fields to the mochi_config.yml file, defaults are created for the other fields

I recently noticed when I defined a single capability in the object, that it no longer took the OTHER capability defaults. In particular, I wanted to only set session expiry interval, but a side effect of this was my maximumReceive becoming 0!

Looking through the code, it is not unreasonable, I just copied the default capability list entirely and changed my 1 or 2 values to get around it. But I wasn't sure if it was worth changing. Does your comment here do something similar? Because in your file config, if you did the same thing, I would have assumed it also would, for example, change maximumReceive to 0 in the absence of it being present.

[dgduncan]

I recently noticed when I defined a single capability in the object, that it no longer took the OTHER capability defaults. In particular, I wanted to only set session expiry interval, but a side effect of this was my maximumReceive becoming 0!

Well if that is the case that is for sure not the intended outcome haha. I will look into that as well.

The config for TLS looks something like this:

type TLS struct {
	Cert   string `yaml:"cert_file"`
	Key    string `yaml:"priv_key"`
	CACert string `yaml:"cacert_file"`
}

So the cert file is referenced via cert_file, key is priv_key, and CA cert is cacert_file
Also from your configuration, to enable TLS on a listener you have to use tls_enabled: true on that listener config just how it is done on the tcp part of this example file

Perhaps that is not the best way to do it; however, my idea is that if you configure TLS, there are many times in which you would not want to add it to every listener, so you have to explicitly set it to true to use.

[SimonMacIntyre]

First, I fixed some typos 🤦
Now I have these 2 segments:

    volumes:
      - ~/.arborxr/certs/xrdm.docker.crt:/etc/ssl/certs/cert.pem:ro
      - ~/.arborxr/certs/xrdm.docker.key:/etc/ssl/certs/key.pem:ro
      - ~/.arborxr/certs/rootCA.pem:/etc/ssl/cacert.pem:ro
      - ./mochi_config.yml:/mochi_config.yml

and in the config:

    tcp:
      port: 8883
      tls_enabled: true
    websocket:
      port: 8882
      tls_enabled: true
    tls:
      cert_file: /etc/ssl/certs/cert.pem
      priv_key: /etc/ssl/certs/key.pem
      cacert_file: /etc/ssl/certs/cacert.pem

Great! Now here is the logs:

2023-09-14 12:03:34 3:03PM INF added hook hook=allow-all-auth
2023-09-14 12:03:34 2023/09/14 15:03:34 INFO Certificate and Key information found in config step=config
2023-09-14 12:03:34 2023/09/14 15:03:34 INFO mochi_config.yml not found
2023-09-14 12:03:34 2023/09/14 15:03:34 INFO defaulting to standard broker configuration
2023-09-14 12:03:34 3:03PM INF added hook hook=allow-all-auth
2023-09-14 12:03:34 3:03PM INF attached listener address=:1883 id=t1 protocol=tcp
2023-09-14 12:03:34 3:03PM INF attached listener address=:1882 id=ws1 protocol=ws
2023-09-14 12:03:34 3:03PM INF attached listener address=:8080 id=stats protocol=http

What I find very confusing:

• it DOES find the mochi_config.yml, we know this because it finds the cert/key info in config, and it starts with the allow-all-auth at the very top
• something goes wrong? mochi_config.yml not found, one line after... presumably reading from that file

I'll see if I can find some human-error on my implementation side

[dgduncan]

@SimonMacIntyre Ahhh yes, that log needs to be replaced with a correct one. I believe what the problem is that it can not find find the TLS files you are referencing in the config.

I just pushed v0.0.2 if you could give that a try. it should be more verbose about the issue.

[dgduncan]

@SimonMacIntyre no rush just wanted to see if you had made any more progress using 0.0.2

[mochi-co]

I've added some thoughts in #285 about how this could be designed to configure hooks 👍🏻

[dgduncan]

@mochi-co What is your opinion on hosting an official image on DockerHub or some other public Docker registry once the basic file based configuration is done? I think it would be very interesting to see the interest in this outside of just wrapping the broker in an embedding application and to get some more visibility. It has been great to get more eyes on the broker recently. I feel like the operational maturity of the broker has grown quite a bit since earlier this year with all of the great feedback from users.

[mochi-co]

@dgduncan Absolutely - I think this is a must have.

[dgduncan]

@mochi-co Would you like to create an official mochi docker account? Sadly organizations are not free anymore in Docker; however, this program looks to be pretty interesting
https://docs.docker.com/docker-hub/dsos-program/

[mochi-co]

FYI - @dgduncan I've submitted an application for the DSOS program

[dgduncan]

Awesome @mochi-co thank you for doing that!

[dgduncan]

@mochi-co Any updates on the DSOS program?

[mochi-co]

@dgduncan We weren't successful in our application because we haven't had an organisation on docker for at least 6 months. This evening I set up an organisation and updated our github workflow to automatically push the latest tag/main. This should work sufficiently for the immediate future, I think.

We will need to evolve our dockerfile and readme appropriately.

It's still a bit of a work in progress, but you can find the docker org and image here:
https://hub.docker.com/r/mochimqtt/server/tags

and pull with docker pull mochimqtt/server

[dgduncan]

That stands to reason. Ok so then in my branch I will target that Dockerfile with my changes, although I do not believe there is much to change. Thank you so much for taking initiative on that!