Some values are not parsed

[mann2107]

I got this response after invoking a chain.

{'text': {'data': {'flight_booking_time_info': [{'departure_from_origin_time': '7 am',
     'return_from_destination_time': '',
     'reach_at_destination_time': '',
     'arrival_at_origin_time': ''}]},
  'raw': 'departure_from_origin_time|return_from_destination_time|reach_at_destination_time|arrival_at_origin_time\n5 pm|7 am|||\n',
  'errors': [],
  'validated_data': {}}}

We can see in "raw" that 5 pm and 7 am are extracted. but in "data" only departure_from_origin_time is outputted.

Here is my Schema:

flight_booking_time_schema = Object(
    id="flight_booking_time_info",
    description="Time-related information about a flight booking request, including departure and arrival preferences.",
    attributes=[
        Text(
            id="departure_from_origin_time",
            description="The preferred time when the traveller wants to depart from the origin location",
            examples=[
                ("I want to depart at 9a", "9a"),
                ("Leaving ideally in the morning", "morning")
            ],
        ),
        Text(
            id="return_from_destination_time",
            description="The preferred time when the traveller wants to return from the destination location",
            examples=[
                ("I prefer returning at 6p", "6p"),
                ("Expected return in the evening", "evening"),                
            ],
        ),
        Text(
            id="reach_at_destination_time",
            description="The preferred time when the traveller wants to reach the destination",
            examples=[
                ("I want to arrive at the Chicago by 5:30 PM", "5:30 PM"),
                ("Prefer to reach by afternoon", "afternoon"),
            ],
        ),
        Text(
            id="arrival_at_origin_time",
            description="The preferred time when the traveller wants to arrive back at the origin location",
            examples=[
                ("I want to come back till 3:00 PM", "3:00 PM"),
                ("Arrival at origin ideally by 7:30 AM", "7:30 AM")
            ],
        ),
    ],
    examples=[
        (
            "I'm looking for a flight departing tomorrow morning and returning at 4:30 PM.",
            [
                {
                    "departure_from_origin_time": "morning", "return_from_destination_time": "4:30 PM", "reach_at_destination_time": "", "arrival_at_origin_time": ""
                }
            ],
        ),
        (
            "I need to arrive at my destination by 6:00 PM tomorrow. Departure should be around 6:30 PM, and return at 9:00 AM next Monday.",
            [
                {
                    "departure_from_origin_time": "6:30 PM", "return_from_destination_time": "9:00 AM", "reach_at_destination_time": "6:00 PM", "arrival_at_origin_time": ""
                }
            ],
        ),
        (
            "I'll need to come back home to Indore on the 25th of February, around 7 am.",
            [
                {
                    "departure_from_origin_time": "", "return_from_destination_time": "", "reach_at_destination_time": "", "arrival_at_origin_time": "7 am"
                }
            ],
        ),
    ],
    many=True,
)

Here is the example I was testing:

Hello there, I hope this message finds you well.
I need to book a business trip. I'll need to fly from Bengaluru on 20th Jan 2024 at around 5 pm.
After the meeting, I'll be returning from LGA on the 25th of Jan 2024, around 7 am.
I would require a hotel and car rental facility also at my destination place.
Please confirm the booking as soon as possible. I would prefer American Airlines with Window Seat in the Outbound.
Also Book me a Aisle seat in Jet Blue in the return Journey. If You can take care that the fare should not be more than $500 that would be great.
Thanks!

[mann2107]

Also when I changed my model from gpt-3.5-turbo to gpt-4. I am getting the following output

{'text': {'data': {'flight_booking_time_info': [{'departure_from_origin_time': '5 pm',
     'return_from_destination_time': '7 am',
     'reach_at_destination_time': '',
     'arrival_at_origin_time': ''}]},
  'raw': 'departure_from_origin_time|return_from_destination_time|reach_at_destination_time|arrival_at_origin_time\r\n5 pm|7 am||',
  'errors': [],
  'validated_data': {}}}

So, with just an adition of '\r' in the raw output, the mapping of output to entity is working fine.

Comaprison Summary:

Erronous:

{'text': {'data': {'flight_booking_time_info': [{'departure_from_origin_time': '7 am',
'return_from_destination_time': '',
'reach_at_destination_time': '',
'arrival_at_origin_time': ''}]},
'raw': 'departure_from_origin_time|return_from_destination_time|reach_at_destination_time|arrival_at_origin_time\n5 pm|7 am|||\n',
'errors': [],
'validated_data': {}}}

Error Free:

{'text': {'data': {'flight_booking_time_info': [{'departure_from_origin_time': '5 pm',
     'return_from_destination_time': '7 am',
     'reach_at_destination_time': '',
     'arrival_at_origin_time': ''}]},
  'raw': 'departure_from_origin_time|return_from_destination_time|reach_at_destination_time|arrival_at_origin_time\r\n5 pm|7 am||',
  'errors': [],
  'validated_data': {}}}

The only differrence I can see in "raw" is an extra "\r" in the output. Thanks

[eyurtsev]

Kor is just a wrapper around an LLM to help write the prompt. Verify that the prompt that it's generated looks good for your application and try to catch an issues (e.g., if the examples are provided in incorrect format).

This extraction task sounds simple and gpt-3.5 will probably be able to handle it without much difficulty.

• Swap your model away from the default gpt-3.5-turbo to gpt-3.5-turbo-0125 -- openAI still hasn't updated the default to the newest model ( https://platform.openai.com/docs/models/gpt-3-5-turbo)
• Take a look at the API to see what the options are: https://eyurtsev.github.io/kor/generated/kor.html#kor.create_extraction_chain
• Try experimenting with switching the encoder (try JSON / CSV)
• Look at these guidelines: https://eyurtsev.github.io/kor/guidelines.html
• See suggestions here: Hallucination even with Four Entities in Schema #261