How to handle limits in ask API?

[WolfgangFahl]

The issues:

WolfgangFahl/py-3rdparty-mediawiki#11

mwclient/mwclient#263

are linked to each other. When querying the ask api with a limit=x parameter I'd expect that the result would be limited. In mediawiki-japi / simplegraph and in the pywikibot based version this seems to work. The offset-based approach in mwclient somehow seems to ignore the parameter. What is the official behavior when using offset and limit via the ask API?

[hexmode]

@WolfgangFahl, I was looking at bugs today and came across your question. Have you tried using a debug log to see what SQL queries are generated? I think I looked into something similar recently and was surprised that SMW did not limit queries.

Since I'm going to be code spelunking, I wonder if you've found any more information about this.

[WolfgangFahl]

@hexmode thank you for looking into this. A the time when I filed the issue we didn't have the tools for more detailed analysis and we have been working around this in the meantime. Using https://pypi.org/project/pymediawikidocker/ it's now much easier to setup an SMW environment with current versions of extensions and go from there and I could indeed try the debugging. Since we have dozens (actually more than 100) details problems currently in our project this one does not have high priority on our list since the other limit of some 10.000 entries per ask query was hitting us much harder and we used the "queryDivision" approach to get around that one thanks to Tim Holzheim's work.

 -qd QUERYDIVISION, --queryDivision QUERYDIVISION
                        divide query into equidistant subintervals to limit
                        the result size of the individual queries

see http://mediawiki-japi.bitplan.com/index.php/Py-3rdparty-mediawiki#wikibackup

This way the queries are now "unlimited" since internally the query is split.