v0.1.2

properly mark static fields

v0.1.1

optimize the chances some google person will try shared key / values …

…and publish results at scale

v0.1.0

release working enwik8 training, thanks to @conceptofmind

0.0.18

oh, jax already has swish

0.0.17

fuse the attention and feedforward input projections, as in paper

0.0.16

fuse the attention and feedforward input projections, as in paper

0.0.15

attention and feedforward are processed in parallel and summed with r…

…esidual

0.0.14

jit by default

0.0.12

fix rmsnorm in palm-lite

0.0.11

fix causal mask value, and add mask value to alibi positional bias to…

… further save