Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net: Include streaming tool call information in model diagnostics #6305

Conversation

TaoChenOSU
Copy link
Contributor

Motivation and Context

Tool call information is currently not included in the model diagnostics when using the streaming APIs.

Description

  1. Record OpenAI tool call information in model diagnostics for the streaming API.
  2. If there is no tool call information, do not record an empty entry.

Contribution Checklist

@TaoChenOSU TaoChenOSU requested a review from a team as a code owner May 17, 2024 00:49
@TaoChenOSU TaoChenOSU self-assigned this May 17, 2024
@markwallace-microsoft markwallace-microsoft added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel labels May 17, 2024
@github-actions github-actions bot changed the title Include streaming tool call information in model diagnostics .Net: Include streaming tool call information in model diagnostics May 17, 2024
@stephentoub
Copy link
Member

Do the other connectors that currently support function calling (Gemini, Mistral) have a similar problem?

@TaoChenOSU
Copy link
Contributor Author

Do the other connectors that currently support function calling (Gemini, Mistral) have a similar problem?

I intentionally did not record the tool calls information in the Gemini connector, since it's going to adapt to the FunctionCallContent and migrate to the new function call abstractions.

@TaoChenOSU
Copy link
Contributor Author

TaoChenOSU commented May 17, 2024

Do the other connectors that currently support function calling (Gemini, Mistral) have a similar problem?

Mistral has the same issue with its streaming API. I will address it in a separate PR. Issue: #6316

@TaoChenOSU TaoChenOSU added this pull request to the merge queue May 17, 2024
Merged via the queue into microsoft:main with commit 1d042be May 17, 2024
15 checks passed
@TaoChenOSU TaoChenOSU deleted the user/taochen/toolcalls-streaming-model-diagnostics branch May 17, 2024 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

4 participants