Callback : checkpoints

[lawofearth]

Firstly, Thank you for contributing this library. Both books and code help me a lot.

I am trying to use SMC2 method to fit my model of interest. However, the methods always take 12 hrs+ for multiSMC with 25 runs which often makes the Python kernel shut itself down. ( I have tried to decrease the N size but I realized that the library can be improved)

It would be nice for people who do not own the permanent computation resource to run their model (including me) and save their state occasionally before their kernel gets kick-off for any reason. By using callback :

The idea of model.fit(epochs=EPOCHS, callbacks=[model_checkpoint_callback]) is came form TensorFlow. Which is a library for Neural networks that often requires high amount of compuational resources and training times.

Tensorflow docs : https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ModelCheckpoint
Example of callback : https://www.pyimagesearch.com/2021/06/30/how-to-use-the-modelcheckpoint-callback-with-keras-and-tensorflow/

[nchopin]

Sorry, I completely missed this. I guess it was during my vacations this summer. I will look into it.