Spacy's NER score

[JorGit91]

Hi,

I am wondering whether Spacy has a method where you can see how much score an entity got when getting detected. For example , if I see " John Smith" on a text string, I'd like to see how much score this entity got when being classified as a PERSON by the model.

My purpose would be to use such a method for dealing with "legible typos" on the entities to be detected ( Example Gabrel Smith could be classified as PERSON)even tho it contains a typo) by adjusting threshold values

[svlandeg]

Hi!

spaCy's NER is a greedy transition-based model, and doesn't calculate scores for the final entities. However, you can use our spancat component for getting span scores (more background on this component can be found in this blog post).

My purpose would be to use such a method for dealing with "legible typos" on the entities to be detected ( Example Gabrel Smith could be classified as PERSON)even tho it contains a typo) by adjusting threshold values

Either way, I don't think that the NER will be a good fit for this type of challenge. It detects entity names by looking at the local context and the token's lexical features. So whether it reads "Sofie" or "Sophie" or "Sofiie" really shouldn't make that much difference to the NER - it's not comparing things against a dictionary or such. So I don't think it will really be bothered by "Gabrel". This is actually the functionality we want: the NER should be able to detect names of things/people that it hasn't seen before.

So in a nutshell I'd suggest trying out spancat (or maybe even textcat) for this type of challenge instead, but even then I'm not sure this will work without relying on some sort of dictionary or (large) language model that specifically knows about common names/strings.