The latest updates on your projects. Learn more about Vercel for Git ↗︎

https://github.com/bytecodealliance/wasmtime/blob/c19c729214e2237902eb177609643cb6523b7f2b/cranelift/native/src/lib.rs#L184

@louisgv The callback to the UI of the fed tokens seams to slowdown the feeding process significantly. (Should be instant)

The callback to the UI of the fed tokens seams to slowdown the feeding process significantly

Yes haha - there's a 42ms artificial lag that I introduced to make the UI a bit more smooth:

ref: https://github.com/louisgv/local.ai/blob/main/apps/desktop/src/providers/thread.ts#L160-L161

For non-accelerated machine and models, this is needed to have something showing :d.......

Currently this copies the cuda dlls next to the local.ai executable if the cargo tauri dev or cargo tauri build command is executed with the --features cublas flag. @louisgv Is this enought to include the dlls into the bundle?

@louisgv Whats the plan going forward on this? Can you take over and handle the auto update stuff?

@LLukas22 yup, I'm on it now!

Kinda want to wait for the Metal fix to land. My main fear with this PR is that the build seems flaky :d.... (OOM?...)

Perhaps we should remove some of the flakyness by building for either cuda or CL only?...

Per the docs, there's still no metal chips on github runner VM yet: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources

So we will still need a metal self-hosted runner I think :d

The last piece missing from this PR is a pipeline to upload the content of each zip artifact into a release.

1. Extract the zip artifact of the 3 jobs (mac, linux, window)
2. Have a 4th job called release, that's in charge of making a release and the update.json
3. Take the path and the signature to form a Tauri update package: https://tauri.app/v1/guides/distribution/updater/#static-json-file
4. Create the draft release