Anomaly detection on ARIMA manifolds

This repository is a code playground for my master thesis at TU Berlin.
The topic of my thesis is Anomaly detection on ARIMA manifolds.

Autoencoder

Example of anomaly detection using generated timeseries data and autoencoder. The example show results for different regularization values.

python3 autoencoderExample.py | tee autoencoder/log.txt

NYC Taxi Dataset

Numenta taxi data

The dataset is downloaded from Numenta.
The raw data is from the NYC Taxi and Limousine Commission.
For other datasets please also see The Numenta Anomaly Benchmark.

python3 taxi_nyc_numenta_data.py | tee results/taxi_nyc_numenta_data/log_taxi.txt

Please find results in results/taxi_nyc_numenta_data

Custom taxi data

The raw data was downloaded from the NYC Taxi and Limousine Commission
and processed into 30 minutes time buckets (see Data processing pipeline).

python3 nyc_taxi_custom.py

Please find results in results/taxi_nyc_custom

Data processing pipeline

Yahoo S5 dataset

Please request and download dataset from yahoo.

python3 yahoo.py | tee log_yahoo.txt

Generated times series

Two time series are generate from two ARMA models with different parameters.
Then they are combined into one time series which contains anomalies. On this time series we use moving windows and train fit an ARMA model on each
window. The parameters of these fitted models are our features.
Finally we use four algorithms to detect anomalies in that feature set
(Robust covariance, One-Class SVM, Isolation Forest,Local Outlier Factor). Please find results in ./results/generated

python3 anomalyTester.py | tee results/generated/log.txt