Deepchecks Monitoring is an open source service 📦 that allows you to monitor machine learning models, get alerts when they fail and perform root cause analysis once a problem was detected.
Deepchecks has a full solution for continous ML validation. Which means that you can use the tests from your testing and CI/CD testing suites as the base for the tests in your monitoring environment. The deepchecks checks (which are the infrastructure of monitors) and suites can be customized and adapted to the monitoring service.
Moreover, when you focus on an issue that requires code level analysis, you can launch the full testing package, with the relevant data on your Python environment and continue the root cause analysis in your "home" environment.
Deepchecks Monitoring provides a comprehensive solution for detecting data and model drift, monitoring key performance indicators (KPIs) 📊, and alerting when anomalies are detected 🔔.
Deploy a hobby instance in one line on Linux/MacOS (Windows is WIP!) with Docker:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/deepchecks/monitoring/main/deploy-oss.sh)"
This will automatically download the necessary dependencies and start the application locally. Installing our open-source service is an excellent way to test the service locally. However, this won't scale to support real-time production usage.
The DeepChecks Monitoring application can be configured by environment variables. The configuration file .env contains settings for the data sources, models, and KPIs that should be monitored.
We would love to welcome new contributors that wish to help this project grow. The most helpful ways to do so are:
- Join our Slack community and talk to us!
- Submit feature requests and bug reports by opening a new issue
- Submit a PR (See our instructions below for Local Development)
The application has the following folder structure:
- docs: Contains the docs for the app
- backend: Contains the backend code
- frontend: Contains the frontend code
Visit the README of each folder to understand more about their structure.
Deepchecks Monitoring open source installation is released under the AGPL 3.0. Please note there is a folder that is not used by default (see backend/deepchecks_monitoring/ee) that is subject to a commercial license (see the license here) and is packaged as part of the repository simply to support upgrading to the commercial edition without downtime.
Using premium features (contained in the backend/deepchecks_monitoring/ee directory) with a self-hosted instance requires a Deepchecks license. To learn more, book a demo or see our pricing page.
Looking for a 💯% open-source solution? Check out the Monitoring OSS repository, which is purged of all proprietary code and features.