Get the One Piece characters info with one click from ONE PIECE WIKI(FANDOM)
Using scrapy to crawl the WIKI page data
This is the PyQt GUI version of one-piece-character-data-scraper, this uses it as a submodule.
You can see what kind of info this get in the link above. Such as character name, height..
Also you are able to sort the data as well.
- PyQt5 (for GUI)
- pandas (for converting)
- Scrapy (for getting the data from the site)
- psutil (for getting the process id to show the log)
- git clone ~
- python -m pip install -r requirements.txt
- go into the onepiece_database directory
- python main.py
First screen, press the "get characters info" button
New dialog will show up to process the getting characters' info from the pages, wait till it finished
You can pause/resume or stop the process. If you press stop button, dialog will be closed.
Result
TODO list
- hyperlink
- crawling in background
- colorful logging
- find/save the log message