Skip to content

Paint2code - a lightweight tool designed to transform your hand-drawn sketches into functional HTML code.

License

Notifications You must be signed in to change notification settings

nico1008/paint2code

Repository files navigation

Paint2Code

Welcome to Paint2Code, a robust tool designed to transform your hand-drawn sketches into functional HTML code. This innovative project leverages advanced image recognition and machine learning algorithms to interpret drawings and convert them into clean, structured HTML elements.

Features

  • Image to HTML Conversion: Upload your sketch and receive HTML code.
  • Support for Multiple HTML Elements: Detects various shapes and interprets them as different HTML elements.
  • Easy to Use Interface: User-friendly interface designed for both beginners and advanced users.
  • Multiple Encoder Models: Utilize various encoder models to enhance accuracy and flexibility in interpreting sketches.
  • Support for Multiple HTML Code Styles: Choose from different HTML coding styles to match your project requirements.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

What things you need to install the software and how to install them:

- Python 3.8 or higher
- matplotlib==3.8.0
- nltk==3.8.1
- numpy==1.26.4
- Pillow==10.3.0
- streamlit==1.33.0
- torch==2.2.1
- torchvision==0.17.1
- tqdm==4.65.0

Installing

A step-by-step series of examples that tell you how to get a development environment running:

  1. Clone the repo:
    git clone https://github.com/nico1008/paint2code
  2. Install the required packages:
    pip install -r requirements.txt

Usage

To use the original Paint2Code dataset, please visit pain2code Dataset to download it.

Model Weights

Data Placement

Place your training data in the .data/all_data folder.

Methods of Use

There are two methods to use this project: via Jupyter notebooks or Python scripts. I suggest using the Jupyter notebooks for better data visualisation.

Method 1: Using Jupyter Notebooks

  1. Data Preparation: Open prepareData.ipynb and run all cells.
  2. Model Training: Open the appropriate notebook for desired model and run all cells:
    • trainCustomCNN.ipynb for CustomCNN
    • trainMobileNet.ipynb for MobileNetV3
    • trainResNet18.ipynb for ResNet18
  3. Model Evaluation: Open the corresponding evaluation notebook and run all cells:
    • evalCustomCNN.ipynb for CustomCNN
    • evalMobileNet.ipynb for MobileNetV3
    • evalResNet18.ipynb for ResNet18

Method 2: Using Python Scripts

  1. Data Preparation:

    python prepareData.py
  2. Model Training:

    python train.py
  3. Model Evaluation:

    python eval.py

Web host

you can visit the paint2code website to use the model on any desired device.

Acknowledgments

I would like to thank the following resources, communities and people for their invaluable contributions and support:

  • The OpenCV team for their robust computer vision library.
  • The PyTorch community for their deep learning framework.
  • KKopilka for major moral support and love.
  • Tony Beltramelli project draws significant inspiration from Tony Beltramelli's pioneering Pix2Code modell.

Contact Me

If you have any questions, suggestions, or need further assistance, please feel free to reach out:


I hope you find Paint2Code useful! If you have any questions or feedback, please feel free to reach out.