Skip to content

gabrielpondc/Coronavirus-Outbreak-Map-Crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

51 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

New Coronavirus Outbreak Map Crawler Project

English | 简体中文

Content


Project Profile

This project was developed on February 16, 2020, initially as a real-time display of the extent of proliferation in China, and the crawlers were sourced from major Internet companies in real time, using PHP and HTML for the front-end of the exhibition

  • Support data display via echart
  • Data Crawl in Mainland China
  • Overseas regional support (all regions except Korea are consolidated data)
  • Data crawling of counties and cities in China (Jiangsu area, other areas just need to change the relevant data sources)
  • Support exporting data for calculation

Run Way

$python time.py

Dependencies

python

pymysql
lxml
selenium(Webdriver[google chrome])
BeautifulSoup

Web Support

Baidu Echart
Apache


Ideas

Crawl the major epidemic display version of the site data, using lxml to structure the site analysis, through the For loop to traverse the fixed format XPATH address

Python crawler->Epidemic website: crawl
Python Crawler->MySql:Crawl through the epidemic site into the database
PHP->MySql:request data
MySql->PHP:Return data

Data source


About the Author