web_scraping_app_demo.mp4

ACC Project: Web Scraping Application

Introduction

The main goal of this project is to build an application designed to monitor website changes, such as price drops and new internship opportunities. Our vision is to create an application that allows users to set preferences and receive real-time notifications through various channels, maintaining a database of updated information for easy and future reference. This is the Result-Focused continuation of the past Learning-Focused project, the legacy repository can be found here.

Teams and Responsibilities

If you are a member of the project, you have been assigned to one of the three teams listed:

Frontend Team

The Frontend Team is tasked with creating a seamless and responsive user interface, ensuring an engaging and accessible experience across different devices.

Backend Team

The Backend Team is in charge of the core application logic, data processing, and the management of the database, ensuring robust and efficient back-end support for the front-end interface.

Web Scraping Team

The Web Scraping Team focuses on the logic behind how to extract data from any website. They are responsible for writing and maintaining scripts that gather data, ensuring the application has the latest content.

Current TODOs:

• Implementing a user authentication system.
• Add scraping functionality and user-specific object
• Implement a sophisticated table for the user's scraped sites dashboard
• Add more pages to enhance frontend UI/UX
• Develop a notification system and interval scraping options
• Deploy the application for public access.

Contribution Guidelines

I welcome any member of this project to contribute to their respective team. Here is a general guideline on how to:

1. Set Up Your Environment:

   • Ensure Git is installed on your machine and your identity is configured.
   • Clone the repository to begin local development.
   • Watch the video posted in the discord on how to set up the database connection

2. Make Changes:

   • Branch off for new features or fixes.
   • Follow the project's coding standards and best practices.
   • Communicate in the discord with other collaborators and team managers

3. Commit Changes:

   • Use clear and descriptive commit messages.
   • Commit often with incremental changes.

4. Push Changes:

   • Push your commits to your branch.

5. Open a Pull Request:

   • Head to the repository on GitHub to open a pull request.
   • Provide a summary of your changes and submit the PR for review.

6. Code Review:

   • Engage with the feedback provided in the review process.
   • Once approved, a manager will merge your changes.

Need Help?

If you're new to Git/GitHub or need a refresher, here are some resources to get you started:

• Git Handbook
• GitHub Flow Guide
• Creating a Pull Request

Here are some team-specific resources that may be helpful:

Frontend:

• Vite
• React
• Bootstrap

Backend:

• Node JS
• Express JS
• MongoDB

Web Scraping:

• Puppeteer
• Chromium & Puppeteer

For additional support, please reach out to via the discord