Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I'm quite new to docker and would like to install this to my Synology NAS #19

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
72 changes: 38 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,62 +1,66 @@
[![Build Status](https://travis-ci.org/vegasbrianc/github-monitoring.svg?branch=master)](https://travis-ci.org/vegasbrianc/github-monitoring)

# A Docker Stack which Monitors your GitHub Repos
Here's a quick start to stand-up a Docker [Prometheus](http://prometheus.io/) stack containing Prometheus, Grafana and [github-exporter](https://github.com/infinityworksltd/github-exporter) to collect and graph GitHub statistics.
# A Docker Stack which Monitors your home network
Here's a quick start to stand-up a Docker [Prometheus](http://prometheus.io/) stack containing Prometheus, Grafana with [blackbox-exporter](https://github.com/prometheus/blackbox_exporter) and [speedtest-exporter](https://github.com/stefanwalther/speedtest-exporter) to collect and graph home network connections and speed.

## Pre-requisites
Before we get started installing the Prometheus stack. Ensure you install the latest version of docker and [docker-compose](https://docs.docker.com/compose/install/) on your Docker host machine. This has also been tested with Docker for Mac and it works well.
Before we get started installing the Prometheus stack. Ensure you install the latest version of docker and [docker-compose](https://docs.docker.com/compose/install/) on your Docker host machine. This has been tested with Docker for Mac and Synology and it works.

## Installation
Clone the project to your Docker host.
# Quick Start

If you would like to change which targets should be monitored or make configuration changes edit the [/prometheus/prometheus.yml](https://github.com/vegasbrianc/prometheus/blob/version-2/prometheus/prometheus.yml) file. The targets section is where you define what should be monitored by Prometheus. The names defined in this file are actually sourced from the service name in the docker-compose file. If you wish to change names of the services you can add the "container_name" parameter in the `docker-compose.yml` file.
If on Mac run this:

## Configuration
In order to pull GitHub stats consistently it is recommended you create a personal access token inside of GitHub. This token will allow you to query the GitHub API more frequently than a public user. [Create GitHub Token](https://github.com/settings/tokens). It is only necessary to give the repo scope to the token permission.
```
git clone https://github.com/maxandersen/internet-monitoring && cd internet-monitoring/prometheus && docker-compose up && open http://localhost:3030/d/o9mIe_Aik/internet-connection
```

otherwise:

<center><img src="https://github.com/vegasbrianc/github-monitoring/blob/master/images/github_token.png" width="600" heighth="500"></center>
```
git clone https://github.com/maxandersen/internet-monitoring
cd internet-monitoring/prometheus
docker-compose up
```

Copy the GitHub Token you created and paste into the bottom of the [docker-compose.yml](https://github.com/vegasbrianc/github-monitoring/blob/master/docker-compose.yml) file under the metrics service section replacing the `GITHUB_TOKEN` with your newly created token.
Goto [http://localhost:3030/d/o9mIe_Aik/internet-connection](http://localhost:3030/d/o9mIe_Aik/internet-connection) (change `localhost` to your docker host ip/name).

The REPOS variable can also be updated to point to the Repos that you wish to monitor. In my example I monitor freeCodeCamp and Docker.
## Configuration
To change what hosts you ping you change the `targets` section in [/prometheus/pinghosts.yaml](./prometheus/pinghosts.yaml) file.

metrics:
tty: true
stdin_open: true
expose:
- 9171
image: infinityworks/github-exporter:latest
environment:
- REPOS=freeCodeCamp/freeCodeCamp, docker/docker
- GITHUB_TOKEN=<GitHub API Token see README>
networks:
- back-tier
For speedtest the only relevant configuration is how often you want the check to happen. It is at 5 minutes by default which might be too much if you have limit on downloads. This is changed by editing `scrape_interval` under `speedtest` in [/prometheus/prometheus.yml](./prometheus/prometheus.yml).


Once configurations are done let's start it up. From the /prometheus project directory run the following command:

$ docker-compose up -d


That's it. docker-compose builds the entire Grafana and Prometheus stack automagically.

The Grafana Dashboard is now accessible via: `http://<Host IP Address>:3000` for example http://192.168.10.1:3000
The Grafana Dashboard is now accessible via: `http://<Host IP Address>:3030` for example http://localhost:3030

username - admin
password - foobar (Password is stored in the `config.monitoring` env file)
password - wonka (Password is stored in the `config.monitoring` env file)

The DataSource and Dashboard for Grafana are automatically provisioned.

If all works it should be available at http://localhost:3030/d/o9mIe_Aik/internet-connection - if no data shows up try change the timeduration to something smaller.

<center><img src="images/dashboard.png" width="4600" heighth="500"></center>

## Interesting urls

Note: replace `localhost` with your docker host ip/name if not running this locally.

The DataSource and Dashboard for Grafana are automatically provisioned. You can still install the dashboard manually if you choose below.
http://localhost:9090/targets shows status of monitored targets as seen from prometheus - in this case which hosts being pinged and speedtest. note: speedtest will take a while before it shows as UP as it takes ~30s to respond.

http://localhost:9090/graph?g0.expr=probe_http_status_code&g0.tab=1 shows prometheus value for `probe_http_status_code` for each host. You can edit/play with additional values. Useful to check everything is okey in prometheus (in case Grafana is not showing the data you expect).

<center><img src="https://github.com/vegasbrianc/github-monitoring/blob/master/images/Grafana_Add_Data_Source.png" width="400" heighth="400"></center>
http://localhost:9115 blackbox exporter endpoint. Lets you see what have failed/succeded.

## Manual Install Dashboard
I created a Dashboard template which is available on [GitHub Stats Dashboard](https://grafana.net/dashboards/1559). Simply download the dashboard and select from the Grafana menu -> Dashboards -> Import
http://localhost:9696/metrics speedtest exporter endpoint. Does take ~30 seconds to show its result as it runs an actual speedtest when requested.

This dashboard is intended to help you get started with graphing your GitHub Repos. If you have any changes you would like to see in the Dashboard let me know so I can update Grafana site as well.
## Thanks and a disclaimer

Thanks to @vegasbrianc work on making a [super easy docker](https://github.com/vegasbrianc/github-monitoring) stack for running prometheus and grafana.

<center><img src="https://github.com/vegasbrianc/github-monitoring/blob/master/images/dashboard.png" width="4600" heighth="500"></center>
I also want to disclaim that Prometheus aren't really (currently) intended for this kind of blackbox/external monitoring and this setup is not in anyway secured. Thus only use this for inspiration and do not blame me if someone hacks this and figure out what your real internet sped is :)

## Troubleshooting
It appears some people have reported no data appearing in Grafana. If this is happening to you be sure to check the time range being queried within Grafana to ensure it is using Today's date with current time.
35 changes: 35 additions & 0 deletions blackbox/config/blackbox.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
modules:
http_2xx:
prober: http
http:
preferred_ip_protocol: "ip4"
http_post_2xx:
prober: http
http:
method: POST
tcp_connect:
prober: tcp
pop3s_banner:
prober: tcp
tcp:
query_response:
- expect: "^+OK"
tls: true
tls_config:
insecure_skip_verify: false
ssh_banner:
prober: tcp
tcp:
query_response:
- expect: "^SSH-2.0-"
irc_banner:
prober: tcp
tcp:
query_response:
- send: "NICK prober"
- send: "USER prober prober prober :prober"
- expect: "PING :([^ ]+)"
send: "PONG ${1}"
- expect: "^:[^ ]+ 001"
icmp:
prober: icmp
2 changes: 0 additions & 2 deletions config.monitoring

This file was deleted.

69 changes: 57 additions & 12 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,35 +22,80 @@ services:
ports:
- 9090:9090
links:
- metrics:metrics
- ping:ping
- speedtest:speedtest
networks:
- back-tier

grafana:
image: grafana/grafana
depends_on:
- prometheus
ports:
- 3000:3000
volumes:
- grafana_data:/var/lib/grafana
- ./grafana/provisioning/:/etc/grafana/provisioning/
depends_on:
- prometheus
ports:
- 3030:3000
env_file:
- ./grafana/config.monitoring
networks:
- back-tier
- front-tier

metrics:
ping:
tty: true
stdin_open: true
expose:
- 9171
- 9115
ports:
- 9171:9171
image: infinityworks/github-exporter:latest
environment:
- REPOS=freeCodeCamp/freeCodeCamp, docker/docker
- GITHUB_TOKEN=<GitHub API Token see README>
- 9115:9115
image: prom/blackbox-exporter
volumes:
- ./blackbox/config:/config
command:
- '--config.file=/config/blackbox.yml'
networks:
- back-tier

speedtest:
tty: true
stdin_open: true
expose:
- 9696
ports:
- 9696:9696
image: stefanwalther/speedtest-exporter
restart: always
networks:
- back-tier

sonos:
tty: true
stdin_open: true
expose:
- 1915
ports:
- 1915:1915
image: maxandersen/sonos_exporter
command:
- '--address=0.0.0.0:1915'
network_mode: host

nodeexp:
privileged: true
image: prom/node-exporter
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
ports:
- 9100:9100
restart: always
command:
- '--path.procfs=/host/proc'
- '--path.sysfs=/host/sys'
- --collector.filesystem.ignored-mount-points
- "^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)"
networks:
- back-tier

3 changes: 2 additions & 1 deletion grafana/config.monitoring
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
GF_SECURITY_ADMIN_PASSWORD=foobar
GF_SECURITY_ADMIN_PASSWORD=wonka
GF_USERS_ALLOW_SIGN_UP=false
GF_INSTALL_PLUGINS=flant-statusmap-panel