Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update recogniser #201

Merged
merged 6 commits into from
Mar 23, 2023
Merged

Update recogniser #201

merged 6 commits into from
Mar 23, 2023

Conversation

mcollardanuy
Copy link
Collaborator

@mcollardanuy mcollardanuy commented Mar 22, 2023 •
edited

In this PR:

  • We're fixing a small tokenization issue with the n-dash.
  • We're allowing a model to be loaded directly from the HF hub (setting load_from_hub to True)
  • We're adding the id2label and label2id mappings as part of the configuration of the NER model. Therefore, we can remove the hard-coded NER mappings. This will allow users to easily use it for a different language (as long as there's NER data for that language)

This was linked to issues Mar 22, 2023
Fix tokenisation problem with n-dash #196
Open
Allow recogniser to directly load NER model #199
Open
Fix hard-coded id2label NER mappings #200
Open
@fedenanni
Copy link
Contributor

Checked n-dash fixed and added test in 3ba38d2

@fedenanni
Copy link
Contributor

Added test for loading model from hub 6529533

@fedenanni
Copy link
Contributor

@mcollardanuy ok, all looks fine, i just added a couple of tests on the things you have added

@fedenanni fedenanni merged commit 5571095 into release Mar 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fedenanni fedenanni fedenanni approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@mcollardanuy @fedenanni