Understand and test language model architectures on synthetic tasks.

Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"

Convolutions for Sequence Modeling

This repo contains data and code for the paper "Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes"

Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena

An open science effort to benchmark legal reasoning in foundation models

Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"