Portuguese words starting with a capital letter are not correctly lemmatized

[dcaled]

How to reproduce the behaviour

>>> nlp = spacy.load('pt_core_news_sm')
>>> doc=nlp('''Feito.''')
>>> for token in doc:
...     print(token.text, token.lemma_, token.pos_)
...
Feito Feito VERB
. . PUNCT

And it is not exclusively when starting a sentence:

>>> nlp = spacy.load('pt_core_news_sm')
>>> doc=nlp('''Eu havia Feito o trabalho.''')
>>> for token in doc:
...     print(token.text, token.lemma_, token.pos_)
...
Eu Eu PRON
havia haver AUX
Feito Feito VERB
o o DET
trabalho trabalhar NOUN
. . PUNCT

It should return "Fazer" as in:

>>> nlp = spacy.load('pt_core_news_sm')
>>> doc=nlp('''feito.''')
>>> for token in doc:
...     print(token.text, token.lemma_, token.pos_)
...
feito fazer VERB
. . PUNCT

Your Environment

• Operating System: Windows
• Python Version Used: Python 3.8.5
• spaCy Version Used: 2.3.5
• Environment Information:

My example includes a verb, but I've also tested using nouns and the same bug occurs.

[svlandeg]

Hi! I'll move this to the discussion board - this issue will be closed but there will be a link/forward to the open thread.

[glangford]

It's 2023 and I came across this problem as well, apparently unresolved.
(Mac, Python 3.11.7, spaCy 3.7.2, Model pt_core_news_lg)

Sentence to reproduce
"Trabalharam com honra e dignidade e estiveram entre os melhores."
(if trabalharam is lower case the lemma is correct)

Two Three questions:

• Why is this bug moved to a discussion, then closed without explanation or a fix
• Do you recommend simply lowercasing text prior to processing with spaCy to work around this problem
• Why is this tagged as a "feature" for the lemmatizer

[glangford]

@svlandeg Any advice on helping the lemmatizer get the right solution in this case?

Portuguese is now using the EditTreeLemmatizer it seems, so there isn't an obvious way to change the lookups, as suggested here for example:

Add rules to Spacy lemmatization
https://stackoverflow.com/questions/63260356/add-rules-to-spacy-lemmatization

Here are two example sentences where the first word (being capitalized) causes the wrong lemma to be created:
"Trabalharam com honra e dignidade e estiveram entre os melhores."
"Fale só um bocadinho sobre o festival"

[svlandeg]

Please consider that spaCy is an open-source toolbox, provided to you for free for any usage (commercial, research or otherwise) and that we spend a LOT of time going through our discussion forums and issue tracker to help users with questions (again, for free) and resolve bugs. Your original post "It's 2023 and I ..." comes across as condescending, which generally is not helpful if you want someone to help you. OS maintenance is a lot of work, and we ask you to be mindful and respectul of that.

To your questions:

* Why is this bug moved to a discussion, then closed without explanation or a fix

This thread wasn't closed. It was moved to a discussion because it's not a bug. This thread is not a bug because errors made by a ML model are not considered to be bugs, as ML models can never be 100% correct or predictable after retraining.

On Github, moving a thread from the Issue Tracker to the Discussion forum can only be done by closing the issue and creating a new thread on the Discussion forum. That's what we did.

* Why is this tagged as a "feature" for the lemmatizer

This thread was tagged as "feat / lemmatizer", indicating this thread is about spaCy's lemmatization feature, which is correct. It helps us group and find similar issues using this internal classification system.

Why is this word not correctly lemmatized in this particular example

Unfortunately ML models will always make mistakes. Often, it would require changing/extending/fixing the training data to fix such behaviour.

Let's have a look at one of the original examples provided by the person who started this topic:

    nlp = spacy.load('pt_core_news_sm')
    doc = nlp("Eu havia Feito o trabalho.")
    for token in doc:
        print(token.text, token.lemma_, token.pos_)

With spaCy 3.7.2, this gives me lemma "Feito" using pt_core_news_sm, but lemma Fazer when using pt_core_news_lg. This indicates that using a larger model will give better accuracy in general (as indicated in the docs). I can't inspect the examples you provided @glangford, as you didn't specify which output you got and what is in fact the correct/expected lemma.

If you want to have more control over the lemmatization, you can consider using a rule-based lemmatizer instead:

> pip install spacy-lookups-data

nlp = spacy.blank("pt")
nlp.add_pipe("lemmatizer")
nlp.initialize()

And follow the advice from the thread you linked.

For the future, incorrect ML predictions can be reported in the following master thread: #3052

[glangford]

Thanks for your reply. My apologies, it certainly wasn't intended to be condescending, I was just expressing confusion about why the issue was unresolved from 2021.

I can't inspect the examples you provided @glangford, as you didn't specify which output you got and what is in fact the correct/expected lemma.

The lemma I receive for "Trabalharam" is the word unchanged, "Trabalharam" using pt_core_news_lg. If I lower case this word the correct lemma "trabalhar" is given. Similarly with "Fale", the lemma is given as "Fale" and lower casing gives the correct lemma "falar".

With spaCy 3.7.2, this gives me lemma "Feito" using pt_core_news_sm, but lemma Fazer when using pt_core_news_lg. This indicates that using a larger model will give better accuracy in general (as indicated in the docs).

I am already using pt_core_news_lg as I mentioned in the original post, and I get these errors.

p.s. Thank you for the pointer to the master thread. This comment demonstrates the same problem in Portuguese with a verb with a capitalized first letter (in this case, "Reserve").
#3052 (comment)

[svlandeg]

Yea, I know you're already using pt_core_news_lg, I just wanted to use that particular example because two different models give two different predictions - thus indicating it's not really a bug but (probably) more of an issue with the underlying data/training procedure.

It does sound like there's some structural problem with first words / capitalized verbs - I wonder whether this simply just doesn't occur as much in the training data. We'll have to look into this, but to manage expectations we probably won't be able to prioritize this in the near future.

Thanks for the post on #3052 - it's good to have these recorded!