NER component in en_core_web_trf doesn't depend on transformer

[frankier]

How to reproduce the behaviour

I just wanted entities so I thought I would only enable NER in case it goes a bit faster.

import spacy
nlp = spacy.load("en_core_web_trf", enable=["ner"])
results = nlp("I went to France for a coffee with Francois")
for ent in results.ents:
    print(ent.text, ent.label_)

It looks like the outputs are just that subsequent bigrams is ORDINAL:

I went ORDINAL
to France ORDINAL
for a ORDINAL
coffee with ORDINAL

The problem goes away when I enable transformer:

import spacy
nlp = spacy.load("en_core_web_trf", enable=["ner", "transformer"])
results = nlp("I went to France for a coffee with Francois")
for ent in results.ents:
    print(ent.text, ent.label_)

Output:

France GPE
Francois PERSON

I suppose ner should depend upon transformer.

Your Environment

• Operating System: Linux
• Python Version Used: 3.11.7
• spaCy Version Used: 3.7.2
• Environment Information:

[svlandeg]

I suppose ner should depend upon transformer.

That's right - it does. In the config file for en_core_web_trf, you can see the following:

[components.ner.model.tok2vec]
@architectures = "spacy-curated-transformers.LastTransformerLayerListener.v1"
upstream = "transformer"

which means that the ner component relies on the transformer component. If you don't enable the latter, the output will be random/nonsensical.

I can see how this behaviour is a bit surprising if you're not used to the config details, especially as no warning is given. It's a bit of an unfortunate side effect of how the listener mechanism works. You might not be interested in all the technical details (🙈) but there's a small "hack" in the code where the listener needs to return "empty" vectors because it actually can't crash at that moment - the same code path can be valid when the model is being initialized and the listener just needs to provide vectors of the right dimension.

That said, I realise how ideally this situation should raise a warning. We'll have a look at how to accomplish that.

[frankier]

I must admit I'm still a bit surprised. Of course a warning would be better, but I can't think of a reason I would want a component to run without all its dependencies running first. Thanks very much for the thorough explanation though!