Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support VOSI 1.1 detail=min #504

Open
msdemlei opened this issue Dec 6, 2023 · 1 comment
Open

Support VOSI 1.1 detail=min #504

msdemlei opened this issue Dec 6, 2023 · 1 comment

Comments

@msdemlei
Copy link
Contributor

msdemlei commented Dec 6, 2023

pyvo currently always parses the full tables endpoint; that can be quite a mouthful, in particular when we are talking about VizieR. But even for GAVO's RegTAP service (http://dc.g-vo.org/tap/tables), that's about 2M of data.

Most of that will be columns of tables any particular user probably will not look at.

To make that a bit more efficient, VOSI 1.1 (https://ivoa.net/documents/VOSI/20170524/REC-VOSI-1.1.html) defines the detail parameter to the tables endpoint. Basically, you get empty table elements when passing detail=min, as in http://dc.g-vo.org/tap/tables?detail=min (which is 146k on the GAVO service at the moment).

As someone accesses metadata of a particular table, you can fill in the columns by requesting tables/table-name.

It would be really nice if dal.vosi.VOSITables could support this. The trouble is that we'd have to put some I/O into the class itself rather than constructing it with the full endpoint content.

Thoughts?

@msdemlei
Copy link
Contributor Author

msdemlei commented Dec 7, 2023

While looking for something unrelated, I noticed that pyVO probably already has all on board to use VOSI detail=min; it looks as if p.dal.vosi.VOSITables supports minimal answers as-is, and there are tests for that, too (see dal/test/data/tap/lazy-table?.xml).

Hence, I think all that's necessary to fix this bug would be to pass a detail=min parameter to the requests in TablesMixin. The way it's designed in VOSI, that ought to be safe to do blindly. We could be particularly conservative and check whether there's a version="1.1" (I suspect TOPCAT does this) on the interface of the ivo://ivoa.net/std/VOSI#tables capability, but I'd say that would be overdoing it.

@msdemlei msdemlei changed the title Support VOSI 1.1 detail=minimal Support VOSI 1.1 detail=min Dec 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant