[Question]: Is log probability available in llm.achat?

[mw19930312]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

I wonder whether log prob can be retrieved through llm.achat (see https://platform.openai.com/docs/api-reference/chat#chat-create-logprobs) for log prob in OpenAI API. If so, is there an example code?

[dosubot]

Hey @mw19930312, great to see you back! 😊 I'm on it - will get back to you with an answer shortly!

[logan-markewich]

@mw19930312 it already is

llm = OpenAI(..., logprobs=True, top_logprobs=20)

response = await llm.achat([ChatMessage(role='user', content='Hello!')])

print(response.logprobs)

20 is the highest openai lets you select.

[dosubot]

Yes, log probability can indeed be retrieved through llm.achat by setting logprobs=True and specifying top_logprobs=20 as shown in the example code provided. This configuration enables the retrieval of log probabilities for the top 20 tokens. The response.logprobs will contain the log probabilities after making a request.

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.