Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type checking for task definition. #693

Open
yoavkatz opened this issue Mar 20, 2024 · 0 comments
Open

Add type checking for task definition. #693

yoavkatz opened this issue Mar 20, 2024 · 0 comments
Assignees

Comments

@yoavkatz
Copy link
Member

yoavkatz commented Mar 20, 2024

Today, tasks fields have no types

FormTask(
inputs=["text", "text_type", "class"],
outputs={"class" , "label"} ,
metrics=[
    "metrics.f1_micro_multi_label", 
    "metrics.f1_macro_multi_label",  
    "metrics.accuracy",
],
 )

This makes it hard for the user to know how to transform their dataset into the right format . They can mistake pass a list for a string, for example and get odd errors or even just unexpected behavior .

We want to add typing definitions:

FormTask(
     inputs={"text":"str", "text_type":"str", "class": "str"},
     outputs={"class": "str", "label":"List[str]"} ,
     prediction_type="str". 
     metrics=[
         "metrics.f1_micro_multi_label",   
         "metrics.f1_macro_multi_label", 
          "metrics.accuracy"
        ]
)

So the FormTask operator will check that all the fields exist and are of the correct type.
It will also check that the prediction_type of the task is compatible with the prediction_type of all the metrics (#667 )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants