Ollama token counts #1179
-
Describe the bugGreat product! easy to setup without the overheads. Thanks! To reproduceNA Additional informationNo response |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 5 replies
-
Hi @aiseei, Ollama returns token counts at the end of the stream or together with the response when not streaming. Just tried this locally and it worked well.
You can then add these token counts to the generation object in Langfuse to track them (docs). |
Beta Was this translation helpful? Give feedback.
-
I have other question , if use streaming , I only get eval_count , but part.prompt_eval_count always undefined for await (const part of chatResponse) {
if (part.eval_count){
output_tokens=part.eval_count
}
.......
} |
Beta Was this translation helpful? Give feedback.
Hi @aiseei,
Tracking token usage when using Ollama works well with Langfuse.
Ollama returns token counts at the end of the stream or together with the response when not streaming. Just tried this locally and it worked well.
Example from the api reference:
You can then add these token counts to the generation object in Langfuse to track them (docs).