Global queryables conflict with STAC Item Search

[m-mohr]

STAC has global Item search at /search. The Filter/CQL support for global Item search requires a /queryables endpoint.

Unfortunately, the link may conflict with https://github.com/opengeospatial/ogcapi-features/blob/master/proposals/search/global-list-of-queryables.adoc which also exposes a /queryables endpoint for /collections.

If I try to implement queryables for /search and /collections via links, I can't distinguish which link applies to /search and which applies to /collections and the endpoint names conflict as well. I assume that both links would be part of the landing page right now.

How can we solve this conflict? Can we enforce that queryables in the landing page applies to /search, queryables in /collections is for /collections? That would solve the link conflict. And then we may just need to be more flexible with the endpoint names and allow more than just /queryables?

cc @philvarner @ycespb

[cportele]

@m-mohr

Can we clarify the issue first?

Part 3 defines the Queryables resource in general (there is no fixed path) and specify the Queryables at /collections/{collectionId}/queryables that apply to the items in collection collectionId.

The proposed Search Part is expected to define (among other resources and operations) a Search resource at /search to execute query that can query the items in all collections and more. For each collection, the queryables of the collection still apply. In addition, the can be a global Queryables resource at /queryables which specifies the subset of the queryables that apply to all collections.

How is that different from the organization of the queryables in STAC?

I also looked at STAC Item Search and it currently does not mention queryables?

Note that there is a difference and that is the POST payload of /search, which we probably should discuss at some point to avoid confusion. The current proposal is here (which includes the md file that you have referenced as a reminder) and there was work in Testbed 18 that has built on that. The Engineering Report has not been published yet, so it is only available on the portal.

[m-mohr]

Sure, thanks @cportele .

The proposed Search Part is expected to define (among other resources and operations) a Search resource at /search to execute query that can query the items in all collections [...] In addition, the can be a global Queryables resource at /queryables which specifies the subset of the queryables that apply to all collections.
How is that different from the organization of the queryables in STAC?
I also looked at STAC Item Search and it currently does not mention queryables?

That's what STAC does, indeed. The global queryables endpoint is defined in the Filter extension, not in Item Search:
https://github.com/stac-api-extensions/filter
I guess a difference is that in STAC there must be a /queryables, and here it "can be". But I guess that's fine.

The current proposal is here

Aha, the proposal is for /search. Good, I think I misunderstood this.

Anyway, the issue that I'm facing right now is that we also implement CQL-based search for /collections. I thought that's also captured here, but may it's not or I did not find it. For /collections we also need a separate list of queryables.
The question is whether we can agree that the link to the collection queryables will be exposed at /collections? And that the endpoint has a "free" name (e.g. /queryables/collections) so that we don't run into conflicts with global search?

I don't think the collection search issue is exclusive to STAC; Records also seem to have somethin similar under (I think) the term "local resource catalog" or so.

If the answer is "yes" for both questions, I think we can quickly close here :-)

[ycespb]

Can't /collections/queryables be used for the list of collection search queryables ? This does not create a conflict with /queryables, nor with /collections/{id}/queryables. Our implementation at emc.spacebel.be is using this URL which can be found via the rel=".../queryables" as well in the /collections response.

[m-mohr]

It may conflict with a potential collection route, so I'd not recommend that by default, but individual implementations could go for that if we leave the path open. A better recommendation might be /queryables/collections. Having the link in /collections is good.

[cportele]

The question is whether we can agree that the link to the collection queryables will be exposed at /collections?

As far as I am concerned: Yes.

And that the endpoint has a "free" name (e.g. /queryables/collections) so that we don't run into conflicts with global search?

I think I would agree with a flexible path since there is no "natural" path. /collections/queryables does not work since the direct sub-resources are the collections. One might also decide to include the queryables in the Collections resource and link to that fragment.

[m-mohr]

Thanks, that works for me!

[pvretano]

This discussion sounds like the Local Resources Catalogue conformance class is OGC API Records.

That conformance class specifies how to extend existing endpoints like /collections or /processes (for example) to make them searchable. So you can say something like GET /collections?q=seabed&bbox=10,10,20,20 and the response would be the normal /collections response except that the only collections included in the response would be those that intersect the specified bbox and which contain the search term seabed is their titles or descriptions.

OGC API Records indicates that the queryables endpoint for local resources endpoints (like /collections or /processes) should be specified via a link (rel=ogc::queryables) in the links section.

As an aside (and this is not part of any specification, its just something I'm experimenting with), my implementation of OGC APIs provides a top-level /queryables endpoint with the following structure:

/queryables
|
+-- /collections
|   |
|   +-- {collectionId}
|       |
|       +-- /items
|       |
|       +-- /styles
|
+-- /processes
|
+-- /jobs

At any endpoint in this tree you get back a JSON-schema that lists the queryables for searching the corresponding endpoint in the OGC API resource tree.

Queyrables endpoint	Target OGC API endpoint	Description
GET /queryables/collections	/collections	List of queryables for searching for collections
GET /queryables/collections/{collectionId}/items	/collections/{collectionId}/items	List of queryables for searching for items of this collection
GET /queryables/collections/{collectionId}/styles	/collections/{collectionId}/styles	List of queryables for searching for styles of this collection
GET /queryables/processes	/collections/processes	List of queryables for searching for processes
GET /queryables/jobs	/jobs	List of queryables for searching for jobs

An empty response means that you can't search the corresponding endpoint in the OGC API resource tree.

The link to queryables in the various links section simply point to these endpoints. So, the link in the /collections object that point to the queryables simply points to /querables/collection.

Anyway, just something I've been playing with in relations to queryables.

[cportele]

I like the idea of making all queryables resources (also) available under /queryables. Would we then add those that are valid for all collections in /search at /queryables/search or still at /queryables as previously planned? I think /queryables/search would be the better fit.

[pvretano]

@cportele yeah, that is what I am experimenting with ... that is making all queryables available under the /queryables. resource.

About queryables across all collections, I was thinking about that too and yes, I think we would put those under /queryables/search.

The basic idea what I have been toying with is that whatever the path is for the searchable resource, if you want the queryables, you just put a /queryables in front of it. So, if the searchable resource is /collections/{collectionId}/items then the queryables can be found at GET /queryables/collections/{collectionId}/items. If the searchable resource is /search then the queryables can be found at GET /queryables/search, etc.

Not sure that notion completely works yet but that is what I have been playing with.

[m-mohr]

Yes, this originates from the Local Resources Catalogue, which we used as a baseline for STAC Collection Search.

The proposal makes sense to me and avoids potential conflicts. Is this meant to be a guideline/recommdation or a requirement? I would prefer it as recommendation, I think. You need to put links anyway, so it doesn't really matter where the endpoints actually are. In STAC we currently use /collections/:id/queryables, which originates from one of the OGC APIs, I think. We can change this in our API extension though.

This structure would also be applied to something like /sortables, right?

[pvretano]

@m-mohr yes, /sortables/collections/{collectionId}, etc...

[cportele]

In STAC we currently use /collections/:id/queryables, which originates from one of the OGC APIs, I think. We can change this in our API extension though.

Features Part 3 will use that path (/collections/{collectionId}/queryables).

The approach with /queryables/... and /sortables/... would be specified in the Schema extension.

The same resource would then be available at /collections/{collectionId}/queryables (Part 3) and /queryables/collections/{collectionId}/items (Part n, Schemas).