MySQL/Percona/MariaDB Edge Cache (MSEC)

msec replicates data from MySQL, Percona and MariaDB, accessed with a Redis API with persistent storage.

Features

• Replica to MySQL, Percona and MariaDB databases using GTID (supporting both MySQL and MariaDB variants).
• In memory cache with expiry, backed by a persistent store using leveled (from riak), based on the concept of log structured merge trees.
• Redis Compatible API, data can be accessed using the hget and hgetall commands.
• Operational instrumentation with sample Grafana dashboards, and Prometheus.

Get Started

Follow the Docker installation instructions to get Docker running on your system.

Clone this repository for the docker compose.yaml with sample data installed.

git clone https://github.com/shortishly/msec.git

Alternatively, with the Github CLI installed use:

gh repo clone shortishly/msec

Change to the newly cloned directory:

cd msec

Start everything up with (MySQL 8.1):

./bin/up

To use MariaDB instead use:

MYSQL_IMAGE=mariadb:11.1 ./bin/up

Or Percona 8:

MYSQL_IMAGE=percona:8 ./bin/up

Operational Instrumentation

msec is instrumented using telemetry with a Prometheus http adapter published to port 9100.

Some sample Grafana dashboards are installed as part of the docker compose, published to port 3000.

Redis API

The keys used by the Redis API are of the form:

database.table.key

In this example, we get all the hash members in the shortishly database, from the grades table, identified by 234-56-7890 (Betty Rubble).

redis-cli hgetall shortishly.grades.234-56-7890
 1) "test4"
 2) "90.0"
 3) "test3"
 4) "80.0"
 5) "test2"
 6) "90.0"
 7) "test1"
 8) "44.0"
 9) "ssn"
10) "234-56-7890"
11) "last"
12) "Rubble"
13) "grade"
14) "C-"
15) "first"
16) "Betty"
17) "final"
18) "46.0"

Or just Betty's grade:

redis-cli hget shortishly.grades.234-56-7890 grade
"C-"

A 'C-' seems harsh, lets give her a 'C' instead:

./bin/db-sql --execute="update shortishly.grades set grade='C' where ssn='234-56-7890'"

The cache is automatically updated:

redis-cli hget shortishly.grades.234-56-7890 grade
"C"