deserializing ask results to native python list of dicts

[WolfgangFahl]

I am the author of the https://pypi.org/project/py-3rdparty-mediawiki/ python library that so far has been using pywikibot as the base library. I also created http://mediawiki-japi.bitplan.com/index.php/Main_Page a few years ago.

Being unhappy with the handling of credentials led to my libraries encoding the credentials in ini files and that handling is compatible between mediawiki-japi and py-3rdparty-mediawiki. Recently i ran into https://phabricator.wikimedia.org/T266935 and since after a few days of debugging i still could not figure out what is going on i decided to try out mwclient instead of pywikibot.

I am quite happy with mwclient and it was much easier to use than pywikibot.

I have already migrated and added some functionality. So there is now https://github.com/WolfgangFahl/py-3rdparty-mediawiki/blob/master/wikibot/wikiuser.py which let's you create credential property files and https://github.com/WolfgangFahl/py-3rdparty-mediawiki/blob/master/wikibot/wikipush.py for pushing pages from one wiki to another including images.

The part that still needs some love is https://github.com/WolfgangFahl/py-3rdparty-mediawiki/blob/master/wikibot/smw.py which can deserialized SMW ask queries. It allows for cut&paste usage of ask queries directly from a wiki and should return a list of dicts (aka table) as a result with native python types. This way e.g. https://pypi.org/project/pylodstorage/ can be used for further processing as e.g. JSON or SQL tables.

That said i'd love to use the ask functionality of mwclient.
unfortunately in the source code for it (see excerpt below) the type information is thrown away which is in the "printrequests" part of the query result.

It would be good if there was a raw-query function that only returns the results without extracting the "answer" yet. With that function i could adopt the deserializer and extend it for more types as outlined in WolfgangFahl/py-3rdparty-mediawiki#5

Shall I create a pull request for this?

Would you like to integrate the deserialize functionality into your library or shall i keep it in mine?

  while offset is not None:
            results = self.raw_api('ask', query=u'{query}|offset={offset}'.format(
                query=query, offset=offset), http_method='GET', **kwargs)
            self.handle_api_result(results)  # raises APIError on error
            offset = results.get('query-continue-offset')
            answers = results['query'].get('results', [])

[WolfgangFahl]

https://pypi.org/project/py-3rdparty-mediawiki/ 0.1.7 is out now it has wikipush functionality as outlined in http://mediawiki-japi.bitplan.com/index.php/Py-3rdparty-mediawiki#WikiPush - the library now supports both mwclient and pywikibot and the credential files are Mediwiki-Japi compatible.

The implementation is based on the code above by refactoring it a bit.
The limit handling issue WolfgangFahl/py-3rdparty-mediawiki#11 also is linked to this issue.

[WolfgangFahl]

The toolkit has been presented at SMWCon Fall 2020 now see https://www.semantic-mediawiki.org/wiki/Wikipush_Toolkit.