Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for customization of webclient for OpenAI #609

Open
deepakn27 opened this issue Apr 20, 2024 · 1 comment · May be fixed by #675
Open

Support for customization of webclient for OpenAI #609

deepakn27 opened this issue Apr 20, 2024 · 1 comment · May be fixed by #675
Assignees
Milestone

Comments

@deepakn27
Copy link

deepakn27 commented Apr 20, 2024

Bug description
Getting Error while doing chatClient.stream(prompt).

This only happens for chatClient.stream(prompt). My finding is it happens due to WebClient
CURL and chatClient.call(prompt) work fine.
Caused by: java.net.UnknownHostException: Failed to resolve 'api.openai.com' [A(1)]

Caused by: io.netty.resolver.dns.DnsNameResolverTimeoutException: [19766: /192.168.29.1:53] DefaultDnsQuestion(api.openai.com. IN A) query '19766' via UDP timed out after 5000 milliseconds (no stack trace available)

Environment
Please provide as many details as possible: Spring AI version - 0.8.1,
Java version - 17,
vector store - SimpleVectorStore (InMemory)

Steps to reproduce
call below method for any prompt.
Prompt prompt = new Prompt("tell me something about java");
return this.chatClient.stream(prompt).map((ChatResponse chatResponse) -> {

        if (!isValidResponse(chatResponse)) {
            logger.warn("ChatResponse or the result output is null!");
            return "";
        }

        AssistantMessage assistantMessage = chatResponse.getResult().getOutput();

        this.chatHistory.addMessage(chatId, assistantMessage);

        return (assistantMessage.getContent() != null) ? assistantMessage.getContent() : "";
    });

Expected behavior
Code should stream proper llm response

Minimal Complete Reproducible example
Prompt prompt = new Prompt("tell me something about java");
return this.chatClient.stream(prompt).map((ChatResponse chatResponse) -> {

        if (!isValidResponse(chatResponse)) {
            logger.warn("ChatResponse or the result output is null!");
            return "";
        }

        AssistantMessage assistantMessage = chatResponse.getResult().getOutput();

        this.chatHistory.addMessage(chatId, assistantMessage);

        return (assistantMessage.getContent() != null) ? assistantMessage.getContent() : "";
    });
@tha2015
Copy link

tha2015 commented May 4, 2024

@deepakn27 I believe that you want to customize WebClient to make it to connect to an HTTP proxy (instead of resolving the IP address of api.openai.com and connect directly to that IP address)? Maybe you should update this issue to make it clear that you need WebClient to be customizable. That will be helpful for closing it.

@deepakn27 deepakn27 changed the title Getting error on return this.chatClient.stream(prompt) Support for customization of webclient for OpenAI May 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants