You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is it possible to add a few symbols in the charset and then finetune with a smaller dataset comprising of these new symbols starting with pretrained weights?
The text was updated successfully, but these errors were encountered:
It should be possible, but you won't be able to use the built-in finetuning code since the output shape will change. The process should look something like this:
Append the additional symbols to the end of the charset (or you could just define a new one with the additional symbols). Make sure to update the charset inside test.py too if you're planning on using that. Also, do check the charset used during validation and update that if necessary.
Manually load the weights for all layers except for the output head (possibly inside train.py just before the training loop)
Use a low learning rate (something like 1e-4 to 1e-3)
Is it possible to add a few symbols in the charset and then finetune with a smaller dataset comprising of these new symbols starting with pretrained weights?
The text was updated successfully, but these errors were encountered: