enhancement of unmatched phenotypes

[harryhoch]

We should have the ability to see the unmatched phenotypes for the profile on the left list, as well as for any column. Basically, if you were interested in comparing a patient against a particular model (as in UDPICS), you want to be able to see what phenotypes do not match against the two, almost as much as those that do.

@harryhoch I wonder if we can think of an approach that is consistent for how to display these regardless if it is on the left list or in a matched column.

This will also be very helpful in particular for debugging ZP matches, don't you think? @cmungall @nlwashington
monarch-initiative/monarch-app#495 for more detail

[yuanzhou]

See #173 for more updates.

[julesjacobsen]

I think this is necessary too. Consider the following case - There are two models, one which matches the given HP terms well and another with no match at all. The second simply vanishes, but it would, as Harry said be very useful to see that there are infact no decent matches. Hiding it is certainly unexpected behaviour. I supose you're doing something else now, but this would be a very good thing to implement if you're still working on phenogrid.

{
    "title": "Two mouse models., one with no matching phenotypes",
    "xAxis": [
        {
            "groupId": "",
            "groupName": "",
            "entities": [
                {
                    "id": "1",
                    "label": "Satb2<tm1Rug>/Satb2<tm1Rug>",
                    "phenotypes": [
                        {
                            "id": "MP:0000088",
                            "term": "short mandible"
                        },
                        {
                            "id": "MP:0000097",
                            "term": "short maxilla"
                        },
                        {
                            "id": "MP:0000102",
                            "term": "abnormal nasal bone morphology"
                        }
                    ],
                    "score": {
                        "metric": "Phenodigm score",
                        "score": 68.72,
                        "rank": 0
                    },
                    "info": [
                        {
                            "id": "Observed phenotypes: ",
                            "value": "",
                            "href": null
                        },
                        {
                            "id": "",
                            "value": "short mandible",
                            "href": "/data/phenotypes/MP:0000088"
                        },
                        {
                            "id": "",
                            "value": "short maxilla",
                            "href": "/data/phenotypes/MP:0000097"
                        },
                        {
                            "id": "",
                            "value": "abnormal nasal bone morphology",
                            "href": "/data/phenotypes/MP:0000102"
                        }
                    ]
                },
                {
                    "id": "2",
                    "label": "Satb2<tm1(cre)Vit>/Satb2<tm1(cre)Vit>",
                    "phenotypes": [
                        {
                            "id": "MP:0008117",
                            "term": "abnormal Langerhans cell morphology"
                        },
                        {
                            "id": "MP:0004776",
                            "term": "vestibular dark cell degeneration"
                        },
                        {
                            "id": "MP:0014052",
                            "term": "increased male germ cell apoptosis"
                        }
                    ],
                    "score": {
                        "metric": "Phenodigm score",
                        "score": 0.00,
                        "rank": 1
                    },
                    "info": [
                        {
                            "id": "Observed phenotypes: ",
                            "value": "",
                            "href": null
                        },
                        {
                            "id": "",
                            "value": "abnormal Langerhans cell morphology",
                            "href": "/data/phenotypes/MP:0008117"
                        },
                        {
                            "id": "",
                            "value": "vestibular dark cell degeneration",
                            "href": "/data/phenotypes/MP:0014052"
                        },
                        {
                            "id": "",
                            "value": "increased male germ cell apoptosis",
                            "href": "/data/phenotypes/MP:0000433"
                        }
                    ]
                }
            ]
        }
    ],
    "yAxis": [
        {
            "id": "HP:0000327",
            "term": "Hypoplasia of the maxilla"
        },
        {
            "id": "HP:0003196",
            "term": "Short nose"
        },
        {
            "id": "HP:0011304",
            "term": "Broad thumb"
        }
    ]
}

[harryhoch]

@julesjacobsen agreed. We've not found any particularly compelling approach to this problem. Let's try to give it some thoughts. Suggestions welcome.

[julesjacobsen]

My expectation would be that there is simply a column containing the label, info and score, but with no coloured squares on the grid.

[harryhoch]

@julesjacobsen, you mean a column for the model with no labels, right? This seems possible, but it's not a great use of screen space. Also, if there are no matches, the score should be very low (if not zero), right?

[julesjacobsen]

Like this:

Yes, the score should be zero, but I still expect there to be a blank column. I wouldn't consider it a waste of space if you want to see that there are no matches. The label is visible and the information for the entity is still present in the mouseover, so it's far from meaningless.

[harryhoch]

Ok. @julesjacobsen, can we get release now and put this on the short list of priority enhancements?

[julesjacobsen]

Sure, go for it.

[harryhoch]

thanks!

[frdougal]

@harryhoch can we close this issue? I think Zhou fixed this in this here: #173. He lists the unmatched phenotypes below the grid.

[jmcmurry]

The unmatched phenotypes can currently be seen when clicked, however, so far users haven't actually been drawn to click there, and have been confused by it. I think the desired behavior is still to have the unmatched things show up on the left with the others. Perhaps with a horizontal line delineating the things unmatched in the query group versus the matched group. A list is better than nothing but it is not as useful, nor as obvious as simply including in the grid from the start.

[frdougal]

I think I understand. What is the proposed behavior? Should I add rows representing the unmatched phenotypes to the bottom of the phenogrid?

[jmcmurry]

Thought I commented but it mustn't have saved?
What do people think of this sort of thing?

At a minimum, the list of phenotypes per matched gene should be segregated between matched/unmatched.

[harryhoch]

@jmcmurry, I think your first response is reasonable, so long as it is collapsiable. @cborromeo, how hard would this be to do?

[frdougal]

I have a question about the first screenshot. Where does hyperglycemia and polyphagia appear if they are matched in some cases but unmatched in others? Is the rule if there is one or more matches then the phenotype appears above the "Unmatched Phenotypes"? Or will all phenotypes be repeated? Once above the Unmatched Phenotypes and once below (assuming all the phenotypes will be both matched and unmatched?

[harryhoch]

I suggest that only those phenotypes that are truly unmatched for any model should go below the line. Others can have a blank square above.

[frdougal]

@jmcmurry are you ok with @harryhoch 's suggestion? If so, I'll figure out how to get it to work.

[jmcmurry]

Agreed with Harry, 1) it should always be collapsible, 2) since there are never phenotypes that are matched all the way across the board, it follows that "unmatched" (in the query column) is by definition a query phenotype with no matches in any target.

As for the unmatches in target columns, I'm conflicted. The simplest thing would be to show only one target column at a time and showing shared unmatches across targets. In case you're wondering, the overlaps shown below in the unmatches are fictional, as are the counts of unmatches. This is just for discussion purposes.

eg.

this should be possible (showing one column at a time, with shared ones at top)	showing multiple columns worth of unmatches at a time may be pointlessly large

Note 3 suggestions:

1. make query entity the first column and differentiate with a line
2. delimiting the unmatched query rows from the target rows somehow. Perhaps another line?
3. any unmatches shared across targets should be at the very top in order to aid comprehension

[harryhoch]

@jmcmurry what goes above the horizontal line? I don't quite understand? Why not just have only the things that are completely unmatched go below the line?

[jmcmurry]

Just so we don't confuse unmatched query phenotypes with unmatched target phenotypes, that's all. If it confuses rather than clarifies, happy to have them all below the line. Did the part about having the query item be a column make sense to you?

[harryhoch]

actually, no - I didn't quite get that. Let me think about unmatched target phenotypes.

[frdougal]

@jmcmurry @harryhoch I spoke to Zhou about this. The execution steps for unmatched phenotypes (in this case query phenotypes) are as follows:

1. take the list of query phenotypes and send them to the simsearch webservice.
2. walk through the data returned and build a list of unmatched query phenotypes by comparing the original list against the ones returned by simsearch.
3. For each unmatched phenotype, run a JSON query to retrieve the label of the phenotype. Please note, phenogrid does not have the label for the phenotypes sent to simsearch. It only has the id (HP:0000006). This call gives phenogrid the label for the phenotype. This call only supports queries for a single phenotype (ex: https://monarchinitiative.org/phenotype/HP:0000006.json). This call is very time consuming if the query returns a large number of unmatched phenotypes. This call also assumed the query parameter is always a phenotype.

I think there is a need for the simsearch webservice to return two lists:

a) the unmatched query items (the list should include the id, label, and data type)
b) the unmatched target items (the list should include the id, label, and data type)

[harryhoch]

@cborromeo, greed. we need a monarch-app call to batch these up, at the very least , and possibly an owlsim call.

Please enter an issue on monarch-app....

[seoanezonjic]

Hi
There is any development about this? I'm very interested in the view as show by @jmcmurry in her post. In my project is very important to show the phenotypes that not match with the query profile in order to check if these mismatches are missing phenotypes when the patient was characterized. I would really like to use your npm package including this option.
Thank you in advance
Pedro Seoane