Fine-tunning huggingface gpt2 on custom plays

[jh-skaion]

Hello team.

i have been trying to fine-tune the gpt2 huggingface model with custom data, which are plays. i have been trying to add this using different tasks but i keep getting errors on format or structure of the file. Can you guide me on how to fine-tune a model using my custom data?

thank you in advance

[klshuster]

Hi there - if you're data can fit in memory, I'd recommend looking at setting up a DialogTeacher in these docs.

If you continue to have issues please post a stack trace and I can take a look to identify any subtle errors.

[jh-skaion]

this is an example of the play. Do i need to change it structure, add or remove something, or can it be used just as it is

[klshuster]

as long as you can format that into an appropriate Message for the an agent to consume, you're good to go.

The DialogTeacher allows you to read and manipulate data in whatever way you deem necessary; the setup_data function simply needs to yield Messages (just wrappers around python dicts) with the following format:

{
    'text': <STRING input to model>
    'labels': <LIST of STRING possible answers/responses (usually a list of 1 element>
}

The function also yields a BOOL indicator implying whether we are at the start of a new "episode" (that is, whether we're still in the same conversation or not); this allows model to build up a dialogue history.

If you're trying to model the "PHYSICIAN" in the play above, your first return value from setup_data may look like the following (truncated for readability):

yield Message({'text': "What a state I am in!...", 'labels': ['If there's life in his fate...']}), True