[Sim plots] Analyze model prediction sanity, via ARIMA-style diagnostic plots of prediction residuals

[trentmc]

TODOs

• Create a unit test for this. Analyze via unit tests on real crypto data.
• Pre-requisite for next step: "Too many plots at once; fix via a tab on left for each group of plots" #1021
• Put into sim_engine plotting

Background / motivation

Parent epic: #1006 "Explore SARIMAX modeling"

Once we've built a model, we analyze the distribution of its residuals = (test data predicted value - test data actual value). There are four complementary diagnostics: raw residuals, histogram, QQ, ACF (correlogram).

How: use one of:

• ARIMA_model.plot_diagnostics() (example).
• statsmodels.graphics.tsaplots.plot_acf, plot_pacf, and ?? for QQ? Example 0: neptune.ai
• NIXTLA StatsForecast library

Prototype

Examples

Example 1: Sikligar thesis

Ref - py notebook

Example 2: PSU

Ref - PSU course, lesson 03

Example 3: plot at different autocorrelation lags

Eg at lags of 1m, 2m, ..., 5m, 10m, ...

Ref ethan rosenthal.com 3 plots above "Reversible Transformations" section

Via this code

import statsmodels.tsa.api as smt

def plot_multi_acf(data, lags, titles, ylim=None, partial=False):
    num_plots = len(lags)
    fig, ax = plt.subplots(len(lags), 1, figsize=(10, 3 * num_plots));
    if num_plots == 1:
        ax = [ax]
    acf_func = smt.graphics.plot_pacf if partial else smt.graphics.plot_acf
    for idx, (lag, title) in enumerate(zip(lags, titles)):
        fig = acf_func(data, lags=lag, ax=ax[idx], title=title);
        if ylim is not None:
            ax[idx].set_ylim(ylim);

    fig.tight_layout();

period_minutes = 5
samples_per_hour = int(60 / period_minutes)
samples_per_day = int(24 * samples_per_hour)
samples_per_week = int(7 * samples_per_day)

lags = [3 * samples_per_hour, samples_per_day, samples_per_week]
titles= ['Autocorrelation: 3-Hour Lag',
         'Autocorrelation: 1-Day Lag',
         'Autocorrelation: 1-Week Lag']

plot_multi_acf(y, lags, titles)