BSP – Phase I Deliverable

Visualizing link structures of web applications

---

1 - BSP’s main objectives

This Bachelor Semester Project as two main objectives.

The first consists in coding a program that its purpose is to go through a website and collect all the different links that it contains. After gathering all the links, the program will then display all the links in a directed graph.

The second involves in learning how to program, but more importantly being able to understand each part of the program as well as testing, debugging and optimising. In this project, Python was the choice for the programming language, and we will take advantage of some libraries to implement in the program.

This program can be later used to see how the structure of a website is and to perceive if there are any clusters. If there are any, maybe then the website can be optimised.

---

2 - Main required competencies before starting the BSP

Before starting this bachelor semester project, a good foundation in Python was necessary. This is because Python is the programming language that will be used to build this project. In conjunction with Python, the basics of HTML and CSS was required to create a website that will be used to test the program.

---

3 - Scientific deliverables of the project

Analysing how the different links appear in a webpage is one scientific aspect of this Bachelor Semester Project. For the second aspect, the student will analyse the actual structure of the corresponding graph.

The fact of analysing the structure of the graph allows the student to ask himself if the graph makes any sense or if the graph can in any way help improving the structure of the website in question.

---

4 - Technical deliverables of the project

Lastly, the technological aspects of this project, cover all the technologies involved to create and manage this project. This project will enhance the student to get familiarised with the different programming tools and it will also allow the student to improve his skills with the Python framework.

The programming tools that are covered in this BSP are the following:

• Learning how to use external Python modules, also known as libraries;

• Get familiar with GitHub and git (very powerful tool for the future since it is commonly used by the companies);

• Get used to time profiling and debugging;

• Using an IDE (Integrated Development Environment).

---

---

REQUIREMENTS To Run the Program

To be able to run this program, you will need to install some libraries which are:

 - requests (in the terminal: pip install requests)
    
 - BeatifullSoup4 (in the terminal: pip install beautifulsoup4)
    
 - networkX (in the terminal: pip install networkx)
    
 - matplotlib (in the terminal: pip install matplotlib)
    
 - spicy (in the terminal: pip intall spicy)