Model saving and loading

VW supports model saving a model and loading it in another VW process. The contents of the model are:

• VW version
• Command line arguments that are marked as keep
  • This is a source of some confusion, the contents of the model will often contain more or less arguments than those provided when originally running VW. This is because non-keep arguments are not saved and some reductions will insert extra command line arguments that themselves are marked as keep
• General VW state
• State for each enabled reduction

Continuing training

When saving a VW model there is a distinction between a model to be used for predictions only or a model that will be used to continue training. By default, loaded models are intended to be able to be used to continue training. Before version 9.0, the default was that loaded models were intended for predicting only. If you wish to save a prediction only model, then the --predict_only_model option must be used. It should be used in both the saving and loading configuration.

Compatibility

In general VW models are not forwards compatible, but they should be backwards compatible.

Command line

Saving a model

-f <file>, --final_regressor <file> is used to save a model file at completion of the training process. This is a binary file.

If you wish to inspect what was saved in the model file there are a couple of options.

--readable_model <file> will save a text file which contains a human readable version of what is saved in the model file.

Version 8.11.0
Id
Min label:-3.35097
Max label:3.13689
bits:18
lda:0
0 ngram:
0 skip:
options: --cbzo --policy linear
Checksum: 1941174661
50664:-0.452471
116060:-0.0388234
130684:0.255139
180474:0.248386
256306:-0.402665

--invert_hash <file> is the same as --readable_model except, it will print the original feature names wherever possible instead of the hash values. The feature names which are filled are the ones which were seen in the data of this run.

Version 8.11.0
Id 
Min label:-1
Max label:0
bits:18
lda:0
0 ngram:
0 skip:
options: --cb_adf --cb_explore_adf --cb_type mtr --csoaa_ldf multiline --csoaa_rank
Checksum: 3590524534
event_sum 1
action_sum 2
:0
Constant:116060:-0.193097
shared^f1:147509:-0.193097
action^f2:257122:-0.193097

Note: invert_hash or readable_model saved files cannot be read back in.

Interpreting the output

Version 8.11.0       
Id 
Min label:-1
Max label:0
bits:18
lda:0
0 ngram:
0 skip:
options: --cb_adf --cb_explore_adf --cb_type mtr --csoaa_ldf multiline --csoaa_rank
Checksum: 3590524534
event_sum 1
action_sum 2
:0
Constant:116060:-0.193097
shared^f1:147509:-0.193097
action^f2:257122:-0.193097

• Checksum is the last line of the general VW state, and is a checksum of the contents up to that point
• All lines between Checksum and :0 are reduction specific state items
• All lines after :0 are model weights in the form feature_name:weight_index:weight_value
• ^ in the feature_name is a namespace

Loading a model

-i <file>, --initial_regressor <file> is used to load in a model file. Since not every configuration option is serialized on the model it is a good idea to use --readable_model to determine what options you need to include with the model load. A common ones is the input format if not using text.

Save tag

If you wish to save a model in the middle of the training process in stead of the end it is possible to process an example with the tag save and VW will interpret this is as a command to save the model file.

save|

Python

When using VW in Python, practically all command line parameters work as expected.

Saving a model

To save a model in Python final_regressor can be used. However, it is important to note that the model saving happens when VW cleans up, so you will need to call finish or destroy the object (with del for example) which will in turn call finish.

You can also call save to save a model file at any point. This only supports saving the binary model file and not the readable version.

vw = pyvw.vw("-f vw.model")
vw = pyvw.vw(final_regressor="vw.model")
vw.save("vw.model")

Loading a model

To load a model file in Python you should use the initial_regressor configuration object when creating the vw instance.

vw = pyvw.vw("-i vw.model")
vw = pyvw.vw(initial_regressor="vw.model")