bug: When there is a large amount of traces, calculating model costs and usage is very slow and ultimately yields no results

[secsilm]

Describe the bug

We have been using langfuse in the production environment, and have generated about 1.65 million traces in the past month. When I check the usage in the past week, the response time is still acceptable.

However, when I select the last month, the "model costs", "model usage", and "user consumption" sections take a long time to load (maybe 5-10 minutes?), and then the loading icon disappears without displaying any results. The CPU usage of postgres also surges.

Loading

CPU useage of postgres container

To reproduce

Generate a large number of traces and then view the dashboard.

SDK and container versions

self-hosting Langfuse: 2.38.0

Additional information

No response

Are you interested to contribute a fix for this bug?

No

[marcklingen]

We’re currently preparing for v3 to address these performance issues of analytical queries: https://github.com/orgs/langfuse/discussions/1902

In the meantime, db IOPS is mostly the bottleneck that you could try to improve to make this faster

[arthurGrigo]

+1

I also have the feeling that there is quite some latency after a prompt chain completed and the full trace is available in the UI.

[marcklingen]

+1

I also have the feeling that there is quite some latency after a prompt chain completed and the full trace is available in the UI.

Interesting, this should not be the case as the SDKs by default flush out events every second. How long do you need to wait?

[arthurGrigo]

Have not measured it but sometimes 1 to 2 minutes for really complex prompt chains.

I should have mentioned that I run langfuse locally using docker-compose.

[marcklingen]

Have not measured it but sometimes 1 to 2 minutes for really complex prompt chains.

I should have mentioned that I run langfuse locally using docker-compose.

thanks for sharing. this will dramatically improve with langfuse v3. In the meantime you could tweak the behavior by increasing the number of threads (docs) as I assume the sdk here is backlogged with events to send to the api.

[arthurGrigo]

Have not measured it but sometimes 1 to 2 minutes for really complex prompt chains.

I should have mentioned that I run langfuse locally using docker-compose.

thanks for sharing. this will dramatically improve with langfuse v3. In the meantime you could tweak the behavior by increasing the number of threads (docs) as I assume the sdk here is backlogged with events to send to the api.

Thanks for the hint!
The docs say one should only use it if really necessary. Are there any known drawbacks or bugs when increasing the number of threads?

[marcklingen]

Have not measured it but sometimes 1 to 2 minutes for really complex prompt chains.

I should have mentioned that I run langfuse locally using docker-compose.

thanks for sharing. this will dramatically improve with langfuse v3. In the meantime you could tweak the behavior by increasing the number of threads (docs) as I assume the sdk here is backlogged with events to send to the api.

Thanks for the hint! The docs say one should only use it if really necessary. Are there any known drawbacks or bugs when increasing the number of threads?

Performance, as it creates additional background threads and might need more time when you try to flush/shutdown (joining the threads). If this is bearable for you right now, you might better wait for v3 or just try how things improve with threads=2