1.4.4

make sure rotary embedding positions are cast to same type as qk

1.4.3

update flag for turning on axial positional embedding and update readme

1.4.2

remove unused keyword argument on Reformer class

1.4.1

make sure rotary embeddings is only applied to original sequence in t…

…he presence of memory key / values and contextual key / values

1.4.0

add and default to rotary positional embeddings

1.2.6

new paper suggests that feedforwards needs to be present

1.2.5

release

1.2.4

Update setup.py

1.2.3

bump

1.2.2

fix post attn layer norm when using embedding factorization