Automatic polling on docker images not working

[pschonmann]

Behaviour

No poller after fresh install

Steps to reproduce this issue

1. Installed latest docker librenms @ synology nas
2. Works fine, scan 192.168.0.0/24 find my devices
3. Periodically not checked.
4. When i log into console and run poller manually, that works

Expected behaviour

Run out of box. Add autodiscovery network, add device, configure and forget.

Tell me what should happen

Add automatic poller, or just info how to set. I didnt find out any info on github wiki

Actual behaviour

Tell me what happens instead

Configuration

• Docker version (type docker --version) : Synlogy package 20.10.3.1233 - Docker version 20.10.3, build b455053
• Docker compose version if applicable (type docker-compose --version) : docker-compose version 1.28.5, build 324b023a
• Platform (Debian 9, Ubuntu 18.04, ...) : Synology DSM 7.0
• System info (type uname -a) : Linux nas 4.4.180+ #41890 SMP Fri Jun 25 02:37:14 CST 2021 x86_64 GNU/Linux synology_geminilake_1520+
• Include all necessary configuration files : docker-compose.yml, .env, ... this is exported config from synology docker package

{
  "cmd" : "",
  "cpu_priority" : 50,
  "enable_publish_all_ports" : false,
  "enable_restart_policy" : false,
  "enabled" : false,
  "entrypoint_default" : "/init",
  "env_variables" : [
     {
        "key" : "PATH",
        "value" : "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
     },
     {
        "key" : "S6_BEHAVIOUR_IF_STAGE2_FAILS",
        "value" : "2"
     },
     {
        "key" : "LIBRENMS_PATH",
        "value" : "/opt/librenms"
     },
     {
        "key" : "LIBRENMS_DOCKER",
        "value" : "1"
     },
     {
        "key" : "TZ",
        "value" : "Europe/Prague"
     },
     {
        "key" : "PUID",
        "value" : "1026"
     },
     {
        "key" : "PGID",
        "value" : "1000"
     },
     {
        "key" : "DB_PASSWORD",
        "value" : "password"
     },
     {
        "key" : "DB_HOST",
        "value" : "mariadb1"
     },
     {
        "key" : "LISTEN_IPV6",
        "value" : "false"
     }
  ],
  "exporting" : false,
  "id" : "74afccc07f32558ba80fb156391e67a88c6c90f737c95c363358a8b1e4c0c4ae",
  "image" : "librenms/librenms:latest",
  "is_ddsm" : false,
  "is_package" : false,
  "links" : [
     {
        "alias" : "mariadb1",
        "link_container" : "mariadb1"
     }
  ],
  "memory_limit" : 0,
  "name" : "librenms-testing",
  "network" : [
     {
        "driver" : "bridge",
        "name" : "bridge"
     }
  ],
  "network_mode" : "bridge",
  "port_bindings" : [
     {
        "container_port" : 162,
        "host_port" : 0,
        "type" : "tcp"
     },
     {
        "container_port" : 162,
        "host_port" : 0,
        "type" : "udp"
     },
     {
        "container_port" : 514,
        "host_port" : 0,
        "type" : "tcp"
     },
     {
        "container_port" : 514,
        "host_port" : 0,
        "type" : "udp"
     },
     {
        "container_port" : 8000,
        "host_port" : 8000,
        "type" : "tcp"
     }
  ],
  "privileged" : false,
  "shortcut" : {
     "enable_shortcut" : false,
     "enable_status_page" : false,
     "enable_web_page" : false,
     "web_page_url" : ""
  },
  "use_host_network" : false,
  "volume_bindings" : [
     {
        "host_volume_file" : "/docker/librenms",
        "mount_point" : "/data",
        "type" : "rw"
     }
  ]
}

Docker info

> Output of command `docker info`

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 11
  Running: 6
  Paused: 0
  Stopped: 5
 Images: 12
 Server Version: 20.10.3
 Storage Driver: btrfs
  Build Version: Btrfs v4.0
  Library Version: 101
 Logging Driver: db
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs db fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ea3508454ff2268c32720eb4d2fc9816d6f75f88
 runc version: 31cc25f16f5eba4d0f53e35374532873744f4b31
 init version: ed96d00 (expected: de40ad0)
 Security Options:
  apparmor
 Kernel Version: 4.4.180+
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.61GiB
 Name: nas
 ID: 4Z37:VEAS:QVVS:MSMQ:22RT:7SSB:XQNS:U43G:YZCM:OLGT:7CNR:46WN
 Docker Root Dir: /volume2/@docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No kernel memory TCP limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
WARNING: No blkio weight support
WARNING: No blkio weight_device support
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Logs

> Container logs (set LOG_LEVEL to debug if applicable)

[murrant]

So, you didn't use docker-compose to deploy the containers? Looks like you are missing at least a poller container.

[ziodona85]

Hi, i have a similar problem.
1 have 3 Dispatcher, they stop to work after about 30 minutes and pollers don't scan any device..
I have done a script that reload all containers as workaround.

#!/bin/bash

cd /opt/Docker/LibreNMS
docker-compose down
docker-compose up -d

After this they return to work correctly.
I have used the rrdcached example in the site.
Can you help me?
Thankks.

[the-d-b]

I have a similar issue. I am new to librenms so I'm sure the issue is on my side. I have followed instructions and have a docker image up. I have only deployed 1 mysql container, and 1 librenms container. No sidecars or additional services as everything looked to be options and I am simply testing.

I can successfully add a device, but no files ever appear in /data/rrd/x.x.x.x directories.

If I connect to the console and manually execute poller.php I do then get some files to appear in the rrd directory.

Discovery works and all my devices show up with some basemetric, even show the device logos correctly.

Occasionally I see a pop-up in the gui letting me know some of my devices have not been polled in the last 15 minutes and when I look at the list of devices, it is all devices I have added.

bash-5.1$ ./validate.php

Component	Version
LibreNMS	21.10.0
DB Schema	2021_25_01_0129_isis_adjacencies_nullable (221)
PHP	7.4.24
Python	3.9.5
MySQL	8.0.27
RRDTool	1.7.2
SNMP	NET-SNMP 5.9
====================================

[OK] Installed from the official Docker image; no Composer required
[OK] Database connection successful
[OK] Database schema correct
[WARN] IPv6 is disabled on your server, you will not be able to add IPv6 devices.
[WARN] Updates are managed through the official Docker image

[gwhipie]

Same issue using the normal poller, I can run it manually and my devices get polled, it just doesn't happen automatically. Is there any solution for this? It makes the current container unusable which really stinks.

[murrant]

It is a requirement you install the dispatcher sidecar, otherwise you have no polling.

[gwhipie]

It is a requirement you install the dispatcher sidecar, otherwise you have no polling.

So the dispatcher sidecar is not optional? It seems to suggest it is, and if I run the script manually it polls, so is there just no cron polling at all? The documentation says "Only used if you enable and run a sidecar dispatcher container" which seems to suggest its optional.

If I run that dispatcher sidecar does that mean I also have to run the REDIS and MEMCACHE containers? I wish the documentation were more clear.

[eoprede]

@gwhipie
I agree, documentation is quite confusing.
After experimenting with it for a bit, it seems that Redis is necessary to enable distributed polling, however Memcache is not needed at all. You will need a dispatcher container. If you look at the compose file, it seems that dispatcher sidecar and main librenms container share the same volume - I am not sure if that's a requirement or not. Logically it should not be, but who knows...

[gwhipie]

@eoprede thanks for your help. So to get it all working properly, I only need the following containers?

librenms
mariadb
dispatcher
redis

Your advice is much appreciated

[eoprede]

@gwhipie I wish I knew the "right" answer, I am just playing around with it right now. As far as I can tell, having these 4 containers will be sufficient to poll the devices and store data, I am not sure about the rest.
I am working on getting it to run properly in k8s, here's what I have working right now and it seems to be able to do basic things: https://github.com/eoprede/librenms-k8s-sample

[murrant]

REDIS is not required if only using one poller and MEMCACHE is never required for the docker container.

Documentation improvements would be great 👍

[selstam]

I have the same problem. I can see the dispatcher docker instance restarting over and over again but doing nothing.

Manual polling using poller-wrapper works in both the librenms and librenms-dispatcher instances.

[mossholderm]

While REDIS may not be a requirement for a single poller, per se, the dispatcher sidecar container won't start unless you have REDIS_HOST defined. Just setting it to localhost (127.0.0.1) seems to be enough to make it start, but I haven't actually managed to get polling to work, even with the dispatcher container up and running.

[xxzk]

@selstam Hi, make sure you have REDIS_HOST=redis in your librenms.env file, if you are using docker-compose.
I have the same problem until docker-compose logs -f shows librenms-dispatcher exit with code 1 and REDIS_HOST env variable needed over and over again.

[Natherul]

Same problem (sort of). I used the example docker compose with the exception that I excluded mariadb (as its cannot be fetched for my rpi) and configured it to fetch from a normal maria instance not running in a container.

By default I could get the dispatcher to fetch data just fine however it does not in my system update any of the graphs making it (for my purpose) useless. For now as a workaround I have added the poller inside the main librenms container and that works. However, if possible it would be nice if the dispatcher would work. Is there any information I can help provide to help track down this issue?

Also before tinkering and just reading the documentation I too was under the impression that the dispatcher was optional so the doc really should be clarified.

[pstaubs]

I also ended up here because the documentation said that the dispatch sidecar and redis services where optional, when in effect they are required to have a usable librenms service. Looks like the documentation is still lacking important installation instructions