v0.3.0

Key Features

This release is based on vllm==0.4.2

What's Changed

• turn off single gpu scenario by @andy-neuma in #88
• Benchmarking : Absolute -> Relative imports by @varun-sundar-rabindranath in #85
• Benchmarking : update Gi_per_thread by @varun-sundar-rabindranath in #90
• Update README.md with sparsity and quantization explainers by @mgoin in #91
• Add notebooks for sparsegpt and marlin compression with nm-vllm by @mgoin in #94
• upstream sync 2024-03-04 by @andy-neuma in #89
• Update README.md by @robertgshaw2-neuralmagic in #96
• Formatting : Fix yapf by @varun-sundar-rabindranath in #101
• Lower unstructured sparsity threshold to 40% by @mgoin in #100
• Benchmarking : Misc updates by @varun-sundar-rabindranath in #95
• upstream merge sync 2024-03-11 by @andy-neuma in #108
• Add lm-eval comparison script by @mgoin in #99
• Benchmarks : Standardize benchmark result store by @varun-sundar-rabindranath in #87
• seed whl centric workflows by @andy-neuma in #116
• Benchmarking : Remote push job by @varun-sundar-rabindranath in #92
• reverted accidental commit to main by @robertgshaw2-neuralmagic in #119
• skipped test for nightly failure by @robertgshaw2-neuralmagic in #120
• Turned back on the Marlin tests by @robertgshaw2-neuralmagic in #121
• Benchmarking : Prepare for GHA benchmark UI by @varun-sundar-rabindranath in #122
• Upstream sync 2024 03 14 by @robertgshaw2-neuralmagic in #127
• Benchmark : Update benchmark configs for Nightly by @varun-sundar-rabindranath in #126
• Benchmark : Modify/Add workflows/actions for github-action-benchmark by @varun-sundar-rabindranath in #123
• Benchmark: fix nightly by @varun-sundar-rabindranath in #131
• Fix nightly - 03/18/2024 by @varun-sundar-rabindranath in #136
• Upstream sync 2024 03 18 by @robertgshaw2-neuralmagic in #134
• Update Dockerfile with extensions support by @mgoin in #107
• Benchmark : Turn-off nightly multi-gpu benchmarks temporarily by @varun-sundar-rabindranath in #130
• Benchmark Fix : Remove special tokens from warmup prompts by @varun-sundar-rabindranath in #140
• Delete .github/pull_request_template.md by @mgoin in #145
• Benchmarking : Update readme by @varun-sundar-rabindranath in #144
• Initial Layerwise Profiler by @LucasWilkinson in #124
• Benchmark Fix : Fix JSON decode error by @varun-sundar-rabindranath in #142
• Upstream sync 2024 03 24 by @robertgshaw2-neuralmagic in #143
• Benchmark : Fix remote push job by @varun-sundar-rabindranath in #129
• Benchmarks : Prune nightly benchmarks by @varun-sundar-rabindranath in #150
• Lock lm-evaluation-harness to commit 262f879 by @mgoin in #151
• Benchmarks : Copy benchmark results to EFS by @varun-sundar-rabindranath in #148
• update readme with nvcc threads option by @varun-sundar-rabindranath in #153
• Generate tarball along with wheel build, and upload both in a package to GH by @dhuangnm in #138
• switch to nightly whl's by @andy-neuma in #154
• whl centric workflow for "remote push" by @andy-neuma in #117
• remove low-workload benchmarks that are flaky by @varun-sundar-rabindranath in #156
• nightly patches by @andy-neuma in #160
• Upstream sync v0.4.0.post1 (merged with upstream-v0.4.0.post1) by @mgoin in #157
• Bump version to 0.2 by @mgoin in #165
• rename wheels to manylinux and remove unused action by @dhuangnm in #167
• Update collect_env.py package list by @mgoin in #169
• Add lm-eval full accuracy sweep using GSM8k by @mgoin in #166
• Upstream sync 2024 04 08 by @SageMoore in #173
• Updated logo in README by @rgreenberg1 in #178
• Fix sparsity arg in Engine/ModelArgs by @mgoin in #179
• rm model_executor/layers/attention directory since it's been moved by @tlrmchlsmth in #181
• Upstream sync 2024 04 12 by @andy-neuma in #183
• mm publish workflow by @andy-neuma in #193
• GCP related build workflow updates by @andy-neuma in #196
• switch to GCP based build VM by @andy-neuma in #201
• cleanup venv by @andy-neuma in #217
• Upstream sync 2024 04 26 by @robertgshaw2-neuralmagic in #211
• update workflows to use generated whls by @andy-neuma in #204
• Fix nightly benchmark scripts by @dbarbuzzi in #229
• Add lm-eval correctness test by @dbarbuzzi in #210
• switch to k8s runners by @andy-neuma in #231
• Upstream sync 2024 05 05 by @robertgshaw2-neuralmagic in #224
• Marlin 2:4 Downstream (for v0.3 release) by @robertgshaw2-neuralmagic in #239
• Misc CI/CD updates by @dbarbuzzi in #240
• bump version to 0.3.0 by @dhuangnm in #241
• [Cherrypick, Bugfix] Fix marlin 2:4 kernel crash on H100 by @mgoin in #245
• [cherry-pick] Update gen-whl.yml by @dhuangnm in #246
• updates for nm-magic-wand, nightly or release (#247) by @andy-neuma in #248

New Contributors

• @SageMoore made their first contribution in #173
• @rgreenberg1 made their first contribution in #178
• @dbarbuzzi made their first contribution in #229

Full Changelog: 0.1.0...0.3.0