You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
afiaka87 edited this page Jun 4, 2021
·
1 revision
Checkpoints made with DeepSpeed ZeRO Stages 2 and 3 will be stored in deepspeed checkpoint folder
As such, they will require DeepSpeed as a dependency in order to resume from or generate with.
See the deespeed conversion script for details on how to convert your ZeRO stage 2/3 checkpoint to a single file.
If using a single GPU, consider running with apex automatic mixed precision instead for a similar speedup to ZeRO.