Livestreaming typing animation #5141 | impact on operational costs chatbots

[HesselWellema]

I have a question

Hi Team,

I noticed the new streaming feature is available and I'm looking forward to implementing it for my customers. However, I have a short question about its feasibility.

We are charged for the use of Directline on a per message basis. Messages are (according to the pricing page) Activity objects being sent to the Bot’s endpoint via HTTP POST or from bot to user.

When a long answer is generated by an LLM, I could wait until it's finished and send it as one HTTP call. Alternatively, I could start streaming right away, which might result in multiple HTTP calls (start, content, finish), or even more.

If every chunk is an HTTP call, I am not sure if this solution will be feasible due to the additional costs (approximately 50 cents per 1000 chunks).

I did notice the line "Livestream is more-or-less 'multiple revisions of the same activity'," but I am not sure how to interpret it.

Thank you.

[compulim]

Rest assured. We designed this with Azure Bot Services team.

The streaming response, no matter how many intermediate chunks are sent, are only count once.

[HesselWellema]

Great. ln that case I will leverage your implementation. Thank you