[Sim, Pdr bot] Can train a model every n epochs, vs every 1 epoch. Enables fancier models, more data

[trentmc]

Background / motivation

Our baseline modeling approach is LinearLogistic, which trains very quickly.

But model-training time can quickly go up by 10x, 100x and even 1000x when:

• fancier models: LinearLogistic -> LinearSVC (10x time) -> xgboost, ffx, etc (100x+ time) -> LLMs (1000x+ time)
• more training data: just 'c' on 1 pair on 1 exchange -> {'o','c','v'} on >1 pair on >1 exchange. Can cause 10x or 100x more data
• cross-validation: 1-fold -> 5-fold -> 20-fold

So we can quickly see a 10x - 1000x increase in model training time.

This will make sim iterations painfully slow, and make it infeasible to run predictoor bots because model building will be too slow.

TODOs / DoD

• In aimodel_ss, have a new setting train_every_n_epochs. Update unit tests, ppss.yaml, etc.
• In sim_engine: can train a model every n epochs
• In predictoor_agent: can train a model every n epochs

[trentmc]

👏