Skip to content

PeterDing/lamb

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 

Repository files navigation

An Implementation for the optimizer LAMB with Tensorflow 2.0

We use pure python code inherted from tensorflow.keras.optimizers.Adam to implement the LAMB algorithm ( Layerwise Adaptive Large Latch optimization).

This implementation uses L2 regularization for all weights excepted for layernorm and all biases.

About

An Implementation for the optimizer LAMB with Tensorflow 2.0

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages