LFX Mentorship 2024-01 Pre-test

[hydai]

Introduction

This discussion serves as a pre-test guideline for mentorship applications related to proposals #3168, #3169, #3170, and #3172. Completion of this pre-test is mandatory for your application to be considered.

Pretest details

1. Framework Execution: Applicants must demonstrate proficiency in building and executing backend frameworks. This involves working with frameworks such as:

   • mlx
   • whisper.cpp
   • intel-extension-for-transformers
   • burn

   You are required to share screenshots and a brief documentation detailing your build and execution process for examples from these frameworks. You can pick any example to demonstrate the execution.

2. For Proposals LFX Mentorship (Mar-May, 2024): Integrate MLX as a new WASI-NN backend #3168, LFX Mentorship (Mar-May, 2024): Integrate Intel Extension for Transformers as a new WASI-NN backend #3169, LFX Mentorship (Mar-May, 2024): Integrate whisper.cpp as a new WASI-NN backend #3170:

   • Follow this guide to build the llama.cpp plugin and execute it with this chat example or this API server example.
   • Select any model to run on your device, then share screenshots and a summary of your build process, model selection, and execution results.

3. For Proposal LFX Mentorship (Mar-May, 2024): Integrate burn.rs as a new WASI-NN backend #3172:

   • Follow this guide to build and execute the rustls plugin.
   • Please choose any example to run the rustls plugin on your device, then share screenshots and a summary of your build process and execution results.

4. Branch Usage:

   • Due to ongoing updates on the master branch, applicants should use the hydai/0.13.5_ggml_lts branch for this pre-test.

Submission Format

Upon completing the pre-test, respond to this discussion with the following information:

Applied Issue: [Indicate the issue number you are applying for]
GitHub Repo: [Provide a link to your GitHub repository containing the completed tasks from sections 1 & either 2 or 3]

For any queries, please post in the designated Q&A thread.

Deadlines

• Pretest Deadline: Submissions must be completed by February 20, 5:00 PM PDT.
• LFX Applicant Deadline: LFX applicants must submit by February 13, 5:00 PM PDT.

[hydai]

Q&A Thread.

Please ask questions in this thread.
If you believe there is a bug, please create a report on the GitHub Issues.

Thanks.

[trkk28097402]

Hi @hydai
I see there is a tps performance test, is there any restriction on parameters (n-predict, n-gpu-layers...) to do this test, or they are all editable?
thx!

[hydai]

Hi @hydai In task 3 for proposal #3172, i'm following the guide that you have specified. (on WSL(Ubuntu)) When i try to run cargo install --path . after building the Rustls plug-in, i get an error saying error: no packages found with binaries or examples. In the target/release folder of the wasmedge_rustls directory, i can see a libwasmedge_rustls.so file. So am i missing any step in between?

@RahulSundkar Please use the libwasmedge_rustls.so in the folder directly. Not sure why this will happen in your environment.

[hydai]

Hi @hydai I see there is a tps performance test, is there any restriction on parameters (n-predict, n-gpu-layers...) to do this test, or they are all editable? thx!

@trkk28097402
Yes, please refer to this options list: https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml#parameters

Here are some examples:

1. https://github.com/second-state/WasmEdge-WASINN-examples/blob/master/wasmedge-ggml/chatml/src/main.rs#L61-L64
2. https://github.com/second-state/WasmEdge-WASINN-examples/blob/master/wasmedge-ggml/embedding/src/main.rs#L18-L34

[trkk28097402]

@hydai
Thank you very much!
I find that there is a wasmedge-ggml-llama-interactive.wasm file, is it still available?
The remaining wasm files under wasmedge-ggml(4 of them)don't appear to have the same function as interactive one have(which show in the wasmegde's twiiter).

[hydai]

@trkk28097402
You can modify the src/main.rs, add any logic you want, and compile it yourself. We just removed these configuration parts to simply the examples. Those configurations are still available.

[Sidhesh2107]

hello, I have successfully built WasmEdge with WASI-NN llama.cpp backend. Could you please guide me on how to execute llama.

[hydai]

Updated in the section. Please move your question to the Q&A thread, and remove this one.

[zxc12523]

Applied Issue: #3168
GitHub Repo: https://github.com/zxc12523/LFX-Mentorship-2024-01-Pre-test

[derekwin]

Applied Issue: #3172
GitHub Repo: https://github.com/derekwin/LFX_3172

[cr7258]

Applied Issue: Integrate whisper.cpp as a new WASI-NN backend
GitHub Repo: https://github.com/cr7258/LFX-Mentorship-WasmEdge-2024-01-Pre-test

[tannal]

Applied Issue: #3170
GitHub Repo: https://github.com/tannal/LFX-Mentorship-WasmEdge-2024-01-Pre-test

[JackySu]

Applied Issue: #3172
Github Repo: https://github.com/JackySu/LFX-Mentorship-2024-01-Pre-test

[guptaaryan16]

Applied Issue #3168
GitHub Repo: https://github.com/guptaaryan16/LFX-WasmEdge-MLX-Pre-Task

[morristai]

Applied Issue: #3172
Github Repo: https://github.com/morristai/WasmEdge-3172-Pre-test

[souravpy]

Applied Issue: #3172
GitHub Repo: https://github.com/souravpy/LFX-Mentorship-WasmEdge-2024-01-Pre-test

[MaryamTaj]

Applied issue: #3170
Github repository: https://github.com/MaryamTaj/LFX-Pre-test
I also wanted to add that completing this pre-test was so much fun! I really enjoyed experimenting with and creating an audio transcription using whisper.cpp - it was so accurate! Thank you so much for this opportunity to learn more about llama.cpp and whisper.cpp :)

[miketang84]

Applied Issue: #3172
GitHub Repo: https://github.com/miketang84/LFX-WasmEdge-3172-Pre-test

[chococandy63]

Applied Issue: #3172
GitHub Repo: https://github.com/chococandy63/LFX-WasmEdge-Pre-test--3172

[XuanDu222]

Applied Issue: #3170
GitHub Repo: https://github.com/XuanDu222/LFX-Mentorship-2024-01-Pre-test-3182

[hax0kartik]

Applied Issue: #3170
Github Repo: https://github.com/hax0kartik/LFX-WasmEdge-Pretest

[IRONICBo]

Applied issue: #3172
Github Repo: https://github.com/IRONICBo/LFX-WasmEdge-PreTest-3172

Thanks! :)

[code-sparsh]

Applied Issue: #3170
GitHub Repo: https://github.com/code-sparsh/LFX-Pre-Test-WasmEdge

[RahulSundkar]

Applied Issue: #3172
GitHub Repo: https://github.com/RahulSundkar/LFX_Mentorship-2024_Pre-Test_3172

[cuticle999]

Applied Issue:#3170
GitHub Repo:https://github.com/cuticle999/LFX-Mentorship-Pre-test-3170

[jaydee029]

Applied under LFX for : #3170
Pretest Repository: https://github.com/jaydee029/LFX-Mentorship-Pre-Test

[treeleaves30760]

Applied Issue: #3170
GitHub Repo: https://github.com/treeleaves30760/LFX_Pre_test

[rum1887]

Applied Issue: #3168
GitHub Repo: https://github.com/rum1887/pretestTaskMLX

[rum1887]

Applied Issue: #3170
GitHub Repo:https://github.com/rum1887/pretestTaskWhisper

[sandeepB3]

Applied Issue: #3172
GitHub Repo: https://github.com/sandeepB3/LFX-Mentorship-2024-PreTest-3172

[estarriol43]

Applied Issue: #3168
GitHub Repo: https://github.com/estarriol43/LFX-MLX-pretest

[estarriol43]

Applied Issue: #3169
GitHub Repo: https://github.com/estarriol43/LFX-itrex-pretest

[ryan-utopia]

Applied Issue: #3168 #3170
GitHub Repo: https://github.com/ryan-utopia/LFX-Mentorship-Pre-test-for-WasmEdge

[uran0sH]

Applied Issue: #3172
GitHub Repo: https://github.com/uran0sH/LFX-Pre-Test-Burn

[ToSeven]

Applied Issue: #3170
GitHub Repo: https://github.com/ToSeven/WasmEdge-2024-01-Pre-test

[TimmyChiang]

Applied Issue: #3170
GitHub Repo: https://github.com/TimmyChiang/LFX-Mentorship-2024-01-Pre-test