Concept search by term and semantic tags

[cristinacivili-work]

Good morning,
I'm writing to inquire about an odd behaviour that we observed in the concept search functionality.

I'm using the /snomed-ct/v3/{path}/concepts/search endpoint and searching by term and semantic tags.
This is what happens:

• search by term "xyz" without semantic tag filter returns a list of concepts (e.g., ConceptA - Disorder, Concept B - Finding)
• adding a semantic tag filter compatible with the results to the same search by term "xyz" (e.g. Disorder) returns an empty list.

This doesn't happen consistently, but only if the term search is hitting a description that doesn't have a semantic tag explicitly defined. But since the semantic tag is also stored on the concept (I see it in the iconId field returned by the search), shouldn't the search hit that field rather than the single descriptions?

Thanks in advance for your help.

[cmark]

Hi @cristinacivili-work,

Thank you for raising your question.

Quickly tested it on our side and it looks fine. I have identified one issue regarding the API, where the semanticTag filter cannot be applied without a non-empty term filter. This will be fixed in the upcoming 7.14.0 release.

My test request was:

POST /snomed-ct/v3/MAIN/concepts/search
{
  "active": true,
  "expand": "fsn()",
  "limit": 10,
  "semanticTag": [
    "disorder"
  ],
  "term": "chest"
}

This yields the proper results, returned concepts are active and has at least one description with the defined semanticTag disorder and with the word chest (full or partial) in the term field.

This doesn't happen consistently, but only if the term search is hitting a description that doesn't have a semantic tag explicitly defined. But since the semantic tag is also stored on the concept (I see it in the iconId field returned by the search), shouldn't the search hit that field rather than the single descriptions?

It is using a search in the description index looking for matching descriptions first (term, semanticTag, language, etc.), then applying an ID filter to the concept search with the concept filter arguments (ECL, ID, active status, etc.).
The iconId field can have two potential type of value, one is an ancestor concept ID which indicates which iconId to use or an actual semanticTag value to use, if that semanticTag value has a corresponding icon in the icon package. That field is never participating in any index search requests.

If you have an example test request that fails we can further investigate the issue, feel free to attach it to this discussion.

Cheers,
Mark

[cristinacivili-work]

Thanks for checking this so quickly @cmark.

I can share an example that I verified on my side:

 {
  "active": true,
  "expand": "fsn(),descriptions(active: true)",
  "limit": 10,
  "term": "Achalasia of oesophagus"
}

This one returns five results, which are procedure and disorders, yet the following one (same with disorder) returns an empty list.

 {
  "active": true,
  "expand": "fsn(),descriptions(active: true)",
  "limit": 10,
  "term": "Achalasia of oesophagus"
  "semanticTag": [
    "procedure"
  ]
}

Is this the expected behaviour?
I understand the second request forces the search to look into the semanticTag field of the actual descriptions, but the behaviour is really counterintuitive if one is using the semantic tag filter to refine a search by terms.

Also, the same search via term "Achalasia of esophagus" and "procedure" actually finds the expected result, but the terms "Achalasia of esophagus" and "Achalasia of oesophagus" are both active descriptions.

[cmark]

Ahh okay, the problem is the British vs American spelling of the word 'oesophagus' and which spelling is selected as FSN on a concept, usually the American is selected and that's why this query fails when you use the British English equivalent.

This would be okay from the description search perspective, but clearly not from the Concept Search API perspective.

The semanticTag filter should filter concepts that have at least one description with any of the listed semanticTag values, however in this case when combined with the term filter, it will only return a concept if it has a description that matches both values.

As you have mentioned, if you try it with the word Achalasia of esophagus and the semantic tag ('disorder' or 'procedure') it works and that could be a workaround for now.
I'll raise a bug report here on GitHub to track the progress of the issue.

Cheers,
Mark

[cristinacivili-work]

Thanks Mark,
I see, it does make sense now. Indeed all the examples of this behaviour that we found involve different American/British spelling.
Another one is term: "Diarrhoea and vomiting" tag: "finding".

I'll keep an eye on the bug report, but please let me know when the fix is released.

Cheers,
Cristina

[cmark]

Hi @cristinacivili-work,

It looks like we will be able to fix this issue in the upcoming major 8.0 release, which is about to be released sometime end of July/early August.

Cheers,
Mark

[cristinacivili-work]

Sounds great, thank you for the update Mark!

Cheers,
Cristina

[cmark]

Hi @cristinacivili-work,

Sorry for my late reply, Snow Owl 8.0.0 has been released a few weeks ago and it fixes the problem described here and in this issue #738.
Let us know if it solves the issue on your end as well.

Cheers,
Mark

[cristinacivili-work]

Thanks @cmark!
We will update soon to the latest version.

Cheers,
Cristina