salsah auto completion returns no or incomplete results

[loicjaouen]

Our users complained about this and I think that it can be reproduced with Anything's test data.

when looking at Things that are linked to other Thing, we find 14 results:

but when we count on the autocomplete to find them out, we find only 4 results:

[loicjaouen]

the request generated by salsah1.5 is:

https://knora-nv.unil.ch/v1/resources?restype_id=http%3A%2F%2Fwww.knora.org%2Fontology%2F0001%2Fanything%23Thing&searchstr=***&numprops=2

and it returns 4 resources. (

When issuing the same request without the numprops option, we get a result with 8 resources.

The documentation says that:

numprops=Integer: Specifies the number of properties returned for each resource that was found (sorted by GUI order)

So it should not change the number of resources found, only the number of properties displayed to the found resources.

The problems probably comes from the sparql query generated by the code in:
https://github.com/dhlab-basel/Knora/blob/develop/webapi/src/main/twirl/queries/sparql/v1/getResourceSearchResultGraphDB.scala.txt

And there is a test for it in ResourcesResponderV1Spec.scala

So let's try to formalise it in code.

[loicjaouen]

Getting a development stack and adding a test

running the tests before modifying them

I get lost everytime doing that, writing it down as notes for myself (and maybe for others).

• starting a graphdb
  I went for the docker solution

  docker run -p 7200:7200 -v $GRAPHDB_LICENSE:/graphdb/GRAPHDB_SE.license -v $KNORA_PIE_FILE:/graphdb/KnoraRules.pie ontotext/graphdb:8.5.0-se -Dgraphdb.license.file=/graphdb/GRAPHDB_SE.license

• initialise it
  from the webapi/scripts folder, run:

  ./graphdb-se-docker-init-knora-test-unit.sh

• run the tests
  on the main folder

  webapi / testOnly org.knora.webapi.e2e.v1.ResourcesV1R2RSpec

---

Notes:
@subotic

• on page https://docs.knora.org/paradox/05-internals/development/graphdb.html
  it is said to run:

  $ docker run --rm -it -v /path/to/license/folder:/external -p 7200:7200 dhlabbasel/graphdb

  it doesn't run as is because there is no tag latest on docker's hub dhlabbasel/graphdb
• does the dhlabbasel/graphdb image does more than adding the PIE file?
  if not it might be simpler to remove dhlabbasel/graphdb and specify on the dock to mount it on /graphdb/KnoraRules.pie
• i think it was mentionned, somewhere in the docs, to push the docker prefs on macOS to 10G or so... I can't find it anymore, but without that, the import of test data always end-up on timeout when reaching incunabula, on my (fast) laptop.

[benjamingeer]

Please don't catch InconsistentTriplestoreDataException, because it's used to indicate a real inconsistency in the triplestore data. If it's reasonable for row.rowMap("properties") not to have a value, use the Map.get method instead:

val maybeProperties: Option[String] = row.rowMap.get("properties")

maybeProperties match {
    case Some(propertiesReturned) => // do something with the properties
    case None => // do something if there are no properties
}

[benjamingeer]

Hi @loicjaouen, I'm just looking at this again because I'm trying to get Fuseki to work again (#1375). I now realise that I don't understand what you mean by using *** as a search string. The Lucene syntax documentation says that you can't use * as the first character of a search string. Also, * can match multiple characters, so what would *** mean?

[mrivoal]

Hi @benjamingeer . @loicjaouen is in holiday, so I will try to answer on his behalf.

In Salsah 1.5, in the advanced search interface when a resource is linked to another with a link property, by using *** in the corresponding field, you get a list of all the possible matches.

In this context, *** returns all the existing records for this property.

In the data entry form, it works in the same way :

It works as well if you use * as the first character of a string:

[benjamingeer]

@mrivoal To get all possible matches, do you have to type ***, or does * work, too?

@tobiasschweizer Do you know anything about this? It seems to work with GraphDB but not with Fuseki, and since The Lucene syntax documentation says that you can't use * as the first character of a search string, I don't understand why it works with GraphDB.

[mrivoal]

The research only works if you enter 3 characters. Only one, be it * or a letter wouldn't return any result.

[benjamingeer]

OK, so maybe we need to treat this as a special case.

[tobiasschweizer]

I think we expect at least three characters. But the handling in v1 and two is not the same. I think in v1 it is handled directly in the sparql template, in in v2, I created a utility function. I would like to unify this.

Currently I am on the train back to Basel from Hamburg. I could look at it next wee if it is still an issue then.

[benjamingeer]

I could look at it next wee if it is still an issue then.

That would be great. I'll be on holiday next week, and I think you know a lot more about Lucene than I do.

[tobiasschweizer]

I think I could spend a whole week on this. Lucene is quite powerful, but preprocessing is needed, e.g., to use AND when submitting several terms separated by space (OR is the default).

You could also make a refactoring issue of Lucene handling in v1 and v2 and assign it to me.

[tobiasschweizer]

Btw: I am your new colleague.

[benjamingeer]

You could also make a refactoring issue of Lucene handling in v1 and v2 and assign it to me.

#1377

I am your new colleague.

Glad to have you back!