[Proposal] Refactoring

[ma7dev]

The majority of repos don't follow the best software engineering practices.

Environment/dependencies packages

• conda used to manage the environment and install essential libraries that are big/core to the package, e.g. TensorFlow, PyTorch, cudatools, etc.
• poetry used to manage dependencies and setup the package

Structure

• .github = github stuff (e.g. github issue templates, github actions workflows, etc.)
• ckpts = checkpoints of pre-trained models that were downloaded
• docs = documentation files
• package_name = the logic for the package
• notebooks = jupyter notebooks for demos
• output = output of running scripts
  • ckpts = checkpoints stored from training
  • figs = figures stored from training/evaluation or any other scripts
  • logs = logs stored from training/evaluation or any other scripts
• scripts = set of scripts to be used to train/evaluate or anything external from the logic of the package
• tests = set of tests to test logics within package_name
• .gitignore = to ignore unwanted files from being pushed to remote branch
• environment.yml = conda environment definition
• install.sh = installing script to setup conda environment and install dependecies using poetry
• mypy.ini = pylint configuration
• pyproject.toml = package definition and list of dependecies to be installed
• pytest.ini = pytest configuration

Commands

• poetry add PACKAGE - to add a package (this will append to pyproject.toml)
  • If the package installation failed and couldn't find another way to add the package, then install it using conda and add to enviroment.yml manually. (leave a comment next to the line)
  • Check on the web for the right channels when install packages using conda
• poetry install - to install the package (package_name)
• act - to run GitHub Actions locally (you will need act installed locally)
• pytest tests - to run all tests manually
• pytest tests/TEST_PATH - to run a specific test file (check pytest documentation for more information)

GitHub Actions - workflows

• build = to test building the project
• tests = to run tests
• publish = to publish the package to pypi
• docs = to generate documentation