Answer streaming

[gyzerok]

Hello and thank you for such a great project!

I've looked through the settings and does not see any ability to enable streaming of the model response. I'd like to see tokens appearing as the model generates them. Is it not a feature available yet?

[justinh-rahb]

This has always been a feature and is the default behaviour. Various proxy and/or tunnelling setups have been found to interfere with this however.

[gyzerok]

Oh, thank you! I am using Caddy, so it's probably about it. Will research it. If you happen to know what to do, I will appreciate yor advice.

Is it done with websockets or sse on the open-webui side?

[justinh-rahb]

SSE