Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetching .xml-dump fails #420

Open
AlexTichy opened this issue Jul 5, 2019 · 0 comments
Open

Fetching .xml-dump fails #420

AlexTichy opened this issue Jul 5, 2019 · 0 comments

Comments

@AlexTichy
Copy link

AlexTichy commented Jul 5, 2019

I'm using the Wikidata Toolkit Examples as they are in online mode, just changed DumpProcessingMode to CURRENT_REVS (any DumpProcessingModes other than JSON produce the same results). Although several online dumps are found, the program fails to download any. This is the console output I get:


*** Wikidata Toolkit: GreatestNumberProcessor


*** This program will download and process dumps from Wikidata.
*** It will scan the dump to find the item with the greatest value
*** for property P1113.
*** See source code for further details.


2019-07-05 17:03:13 INFO - Using download directory C:\my...\dumpfiles\wikidatawiki
2019-07-05 17:03:13 INFO - Found 0 local dumps of type FULL: []
2019-07-05 17:03:14 INFO - Found 7 online dumps of type FULL: [wikidatawiki-full-20190701, wikidatawiki-full-20190620, wikidatawiki-full-20190601, wikidatawiki-full-20190520, wikidatawiki-full-20190501, wikidatawiki-full-20190420, wikidatawiki-full-20190401]
2019-07-05 17:03:16 WARN - Could not find any dump of type FULL.
2019-07-05 17:03:16 INFO - Finished processing.
2019-07-05 17:03:16 INFO - Processed 0 entities in 0 sec
Found 0 matching items after scanning 0 items.
No number with a specified value found yet.

It seems that the error is in WmfOnlineStandardDumpFile's method FetchIsDone(). It reads through the md5 checksum files of the dumps and attempts to find a line ending on "-pages-meta-history.xml.bz2". However, this specific line ending doesn't exist in the files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant