Releases: lucidrains/soft-moe-pytorch
Releases · lucidrains/soft-moe-pytorch
0.1.8
0.1.7
remove erroneous split by rank backwards
0.1.6
Full Changelog: 0.1.5...0.1.6
0.1.5
add option to use layernorm instead of rmsnorm
0.1.4
expose an extra flag
0.1.2
another quick fix
0.1.1
add a contiguous, thanks to @conceptofmind
0.1.0
add ability to noise gates and anneal noise
0.0.9
just port over some logic from st moe
0.0.8
quick fix