Skip to content

Find out how food's price changes over time with our data visualization tool. Powered by a webscraper tool that collects up-to-date data from supermarkets's websites (daily)

License

e-llo/ellucidata

Repository files navigation

ellucidata

This project contains an efficient webscraping tool (Selenium) and data visualization dashboards.

Find out how food's price changes over time with our data visualization tool. Powered by a webscraper tool that daily collects up-to-date data from supermarkets's websites!

image

ellucidata.mp4

📖How does it work?

We developed an algorithm capable of scraping a website's interface, collecting data of a list of products in many supermarkets and send to a cloud MongoDB. Also, some dashboards were developed (using HMTL, CSS and Javascript) to allow analysis on products' price changes over time.

🧰Languages and Tools

  • Python
  • MongoDB
  • Selenium
  • JavaScript
  • NodeJS

⁉️Why?

Our goal is to collect data from online supermarkets' products in order to analyse price changes during some period of time.


How can I run this locally?

💻 Prerequisites

Scraper

  • Python3
  • Chrome Driver (make sure that the driver's version matches Google Chrome's version in your computer)

Data Visualization

☁️ Clone this repository

https://github.com/e-llo/ellucidata.git

🌿 Install Dependencies

On the command line (Windows OS):

Scraper

python -m pip install -r requirements.txt

Visualization

npm install
npm run env

🏃‍♀️ Run

On the command line:

Scraper

python insert_data.py

Visualization

npm start

After the application is running, access http://localhost:1337/ from your browser


👩‍💻👨‍💻👨‍💻 Contributors

Julia Rolemberg profile picture
Julia Rolemberg
Pedro Heck profile picture
Pedro Heck
Raphael Menezes profile picture
Raphael Menezes de Jesus

About

Find out how food's price changes over time with our data visualization tool. Powered by a webscraper tool that collects up-to-date data from supermarkets's websites (daily)

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published