New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wrapping custom application endpoint not working #1914
Comments
Hello, In this case it seems that you only set up the call to If you want to integrate calls to your custom URL for the scan you need to use the import os
from dataclasses import asdict
from typing import Sequence, Optional
import giskard
from giskard.llm import set_default_client
from giskard.llm.client import LLMClient, ChatMessage
import requests
import pandas as pd
class MyApiClient(LLMClient):
def complete(
self,
messages: Sequence[ChatMessage],
temperature: float = 1,
max_tokens: Optional[int] = None,
caller_id: Optional[str] = None,
seed: Optional[int] = None,
format=None,
) -> ChatMessage:
# In here I assume that your API have the same format as OpenAI, adjust based on your needs
completion = requests.post('https://api-url.com/v1/chatbot/request', json={
'model': self.model,
'messages': [asdict(m) for m in messages],
'temperature': temperature,
'max_tokens': max_tokens,
'seed': seed,
'response_format': format
}, headers={'Content-type': 'application/json'}).json()
self.logger.log_call(
prompt_tokens=completion['usage']['prompt_tokens'],
sampled_tokens=completion['usage']['completion_tokens'],
model='gpt-4',
client_class=self.__class__.__name__,
caller_id=caller_id,
)
msg = completion.choices[0].message
return ChatMessage(role=msg.role, content=msg.content)
my_api_client = MyApiClient()
set_default_client(my_api_client)
def call_my_api(df: pd.DataFrame):
return [my_api_client.complete(message).content for message in df['messages']]
# Create a giskard.Model object. Don’t forget to fill the `name` and `description`
giskard_model = giskard.Model(
call_my_api, # our langchain.LLMChain object
model_type="text_generation",
name="My Generic Assistant",
description="A generic assistant that kindly answers questions.",
feature_names=["messages"],
)
scan_results = giskard.scan(giskard_model)
display(scan_results) # in your notebook |
Thanks @kevinmessiaen. we figured out the model wrapping issue, and the scan initiates but fails with the error below: We are using the Azure OpenAI LLM for the detector. |
Issue Type
Bug
Source
source
Giskard Library Version
2.11.0
Giskard Hub Version
N/A
OS Platform and Distribution
No response
Python version
3.11.4
Installed python packages
No response
Current Behaviour?
Standalone code OR list down the steps to reproduce the issue
Relevant log output
The text was updated successfully, but these errors were encountered: