Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Develop #442

Open
wants to merge 63 commits into
base: main
Choose a base branch
from
Open

Develop #442

wants to merge 63 commits into from

Conversation

philpax
Copy link
Collaborator

@philpax philpax commented Nov 12, 2023 •
edited

The pending PRs were interrelated, but I didn't want to leave main in a half-working state, so I've merged all the PRs into a new develop branch. The plan is to work on this branch and leave main in maintenance mode until this is ready.

Closes #365, closes #403, closes #439, closes #77.

This integrates:

  • a GGML version upgrade
  • GGUF support
  • BERT support
  • APIs for context-shuffling

This is the to-do list:

  • Update to the latest GGML
  • Fix CUDA inference
  • Fix OpenCL inference
  • Fix Metal inference
  • Fix the embedded tokenizer
  • Readd quantisation
  • Modularize the model definitions (i.e. move block inference to the block struct)
  • Fix models (ensure they're uncommented in llm):
    • Fix BLOOM
    • Fix GPT-NeoX
    • Fix Falcon
    • Fix GPT-2
    • Fix GPT-J
    • Fix MPT
    • Fix BERT
  • Remove the expects
  • Fix the TODOs

nerdypepper and others added 30 commits August 7, 2023 14:55
@nerdypepper @LLukas22 @philpax
add bert model
7a10cfb 
Co-authored-by: Lukas Kreussel <lukaskreussel@gmail.com>
Co-authored-by: Philpax <me@philpax.me>
@philpax
refactor: move ggml format to module
ffb0519
@philpax
fix(ggml): use byte-arrays for magic
d5c2562
@philpax
feat(ggml): impl unwired gguf loader
dd7aa26
@Andreybest
Remove error on context window overflow
b9b1391
@philpax
feat(gguf): gguf-v2 support
e166b7c
@philpax
chore(gguf): clippy fix
90c6797
@philpax
Merge branch 'main' into gguf
ddf4e40
@philpax
fix(gguf): drop the null terminator
38dd730
@philpax
refactor(ggml): begin decoupling the old formats
41462ed
@philpax
feat(bin): add gguf-explorer as debugging tool
2de2df7
@philpax
refactor(gguf): split metadata out + use macros
0da661f
@philpax
wip: rewrite loader to use GGUF; almost wire up llama
e182444
@philpax
fix(cli): use info log level
178a0fb
@philpax
wip: successfully load a llama2 gguf*
2a9417a 
* with some heavy caveats, see the PR
@philpax
wip: disable everything that's broken
823828d
@philpax
feat(llama): validate tensor data layout
eb8c508
@Andreybest
Add "context swap" functions to session and add "decoded_tokens" to s…
99a9fb4 
…napshot read/write
@LLukas22
Build a against newer GGML version
6835335
@LLukas22
Update llama-cpp
1eb0d79
@LLukas22
Include ggml-alloc.c during build
ad136e1
@LLukas22
Merge remote-tracking branch 'upstream/main' into feat/ggml-update
fd3ff64
@LLukas22
Hopefully fix linux build
ab381c7
@LLukas22
Remove Scratch Buffers
4ebb16e
@LLukas22
Use GraphAllocator in LLaMA architecture
995dd79
@LLukas22
Working graph allocator for llama
6ba5126
@LLukas22
Scope input_length and session_len to BuildContext
78b0e25
@LLukas22
Logging + mpt tests
8ad589b
@LLukas22
Try to set the cuda scratch offset
e506b0b
@philpax
feat(llm): remove architecture param
f398ebd
@philpax
wip(llm): reconvert ggml tokenizer with GPT-4
58cb8cc
@philpax
Merge branch 'main' into gguf
43ebc3d
@philpax
fix(ggml/llmb): use IndexMap for GGUF
e4db5b9
@philpax
fix(llmb): disable embedded tokenizer
8996061
@philpax
refactor: move loading logic back into llmb, simplify
34379ac
@philpax
feat: implement GGUF write / llm gguf rebuild
a4bbdbf
@philpax
feat(llm): implement gguf add-hf-tokenizer
df1aa0e
@philpax
fix(gguf): add support for ggufv3
d5e7b61
@philpax
fix(gguf): load bools correctly
5457414
@philpax
feat(llm): get GPT-NeoX loading again
6114076
@philpax
feat(llm): get GPT-NeoX closer to working
8961ff7
@philpax
feat(llm): more attempted GPT-NeoX fixes
be709ed
@philpax
fix(cli): in info, elide known large items
a728852
@philpax
fix(llmb): usercallback show error
5ed38be
@philpax
fix(ggml): bindgen issues
fcbfb4d
@philpax
Merge pull request #398 from nerdypepper/add-bert-arch
52c2bb6 
add bert model
@philpax
Merge branch 'develop' into feat/ggml-update
5e4b35f
@philpax
Merge pull request #428 from LLukas22/feat/ggml-update
e5e0fe1 
Build against newer GGML version
@philpax
Merge branch 'develop' into andreybest-main
2e3c6f7
@philpax
Merge pull request #424 from Andreybest/main
2c127df 
Add "context swap" functions to session and add "decoded_tokens" to snapshot read/write
@philpax
chore: fix precommit
eddf953
@philpax
Merge in develop
ab956c9
@philpax
chore: fix precommit
7c3d1cf
@philpax
Merge branch 'develop' into gguf
4401631
@philpax
Merge pull request #412 from rustformers/gguf
535eda1 
GGUF support
@philpax
Merge branch 'main' into develop
d318da1
@philpax
wip: update ggml crate only
bd63f63
@philpax
Start porting llama.cpp + simple.cpp to Rust
ce927dd
@philpax
feat(lcsc): start implementing llama functions
ab85bf6
@philpax
chore: update llama-cpp
00aaf4a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@philpax @nerdypepper @Andreybest @LLukas22