Stream chat response in Assistants

[onerinas]

This PR is an attempt to make streaming working in Assistants.

Feel free to create a new PR or push to the same branch if you want to try different implemention.

With the current changes, usage looks like this:

llm = Langchain::LLM::OpenAI.new(api_key: ENV["OPENAI_API_KEY"])

thread = Langchain::Thread.new

Langchain::Assistant.new(llm: llm, thread: thread) do |response_chunk|
  print(response_chunk["delta"]["content"])
end

Screenshot of the same logged in a Sidekiq worker:

[andreibondarev]

@onerinas Looks like it works for streaming tool_calls as well if you yield:

print(response_chunk["delta"]["tool_calls"])

Are you planning on finishing this PR?

[onerinas]

@andreibondarev That’s good to know it works for tool_calls as well.

What would be the next steps to make some progress on the PR? I assume these?

• yield in tool_calls
• Update usage of Assistant in the methods docs and README
• Specs

I can give it a try. I’m okay with you picking it up as well if you want.

[andreibondarev]

@onerinas Go for it!