Onlineweb 4

Contributing

Development environment

The by far easiest way to start developing is to use Visual Studio Code (VS Code) with remote-containers.

After starting the container you can run the following to run the test-suite:

poetry install
poetry shell

npm ci
npm run build

# run tests
pytest

# install pre-commit hooks to automatically check formatting
pre-commit install

# to run
./manage.py migrate

./manage.py runserver

# OnlineWeb4 should be available on localhost:8000

Authentication

Authentication goes through our Auth0-tenant, to get access to development configuration contact dotkom@online.ntnu.no.

Frameworks

Frontend

Frontend code is located in the assets folder. It's a mixture of old jQuery code and newer React code. The code is bundled with esbuild.

Backend

Python 3 with Django is used as a backend and is mostly located in the apps folder. Django templates are in templates folder. The API uses Django REST framework

To find the list of available commands when using python manage.py (alternatively ./manage.py), see the docs.

Installation - Git and repository setup

Setting up your Git user and cloning the repo

We recommend using GitHub's own Command Line Interface (CLI):

gh auth login
gh repo clone dotkom/onlineweb4
cd onlineweb4

Normal flow for making a PR:

git switch -c new-feature
# exciting code
py.test
# can use `git add -ip`
git add <files>
git commit
gh pr create

Local installation

To run the project you need Python 3.12 and node >=20 for bundling the frontend-dependencies.

We recommend using either pyenv or rye to manage your Python-versions. You will need Poetry to install the dependencies, we recommend using pipx to instal Poetry. Using the devcontainer above automatically install the correct python-version and has pipx and poetry pre-installed.

For Node you can use NVM.

Testing and linting

We use Ruff for linting and formatting. To run the linting run ruff check .. You can add git hooks that automatically check the linting when committing with pre-committ install.

pre-commit install

And run the lints manually by running

pre-commit run --all-files

To run the tests you can call

# most tests using Django templates require the `webpack-stats.json` to exists
npm run build

py.test

Which should work as long as you have properly set up the Python environment.

Creating migrations

After doing changes to a model, you will need to make an migration for the database-scheme. You can automatically create those commands by running the following:

python manage.py makemigrations

Note that migrations should be properly formatted and linted as other files, but should be fixed with our pre-commit hooks.

Continuous Integration and Continuous Delivery

Pushes made to the main branch will trigger a redeployment of the application on dev.online.ntnu.no.

Pull requests trigger containerized builds that perform code style checks and tests. You can view the details of these tests by clicking the "detail" link in the pull request checks status area.

Important: We have integration tests with Stripe that require valid test-API-keys, those tests are not run by default locally, or when creating a PR from a fork of this repository. To run them, first get ahold of the appropriate testing keys, then add them to an .env file in the project root, the name of the environment variables are in .devcontainer/devcontainer.env.

Manual release

Zappa

The project is deployed to AWS Lambda with the use of Zappa. To deploy (should be done automatically), build a Docker image and push it to AWS ECR. Then you can run zappa update <stage> -d <docker-ecr-image>. You'll have also have to build NPM and deploy static first if this has been changed since last deploy.

Example for prod

# create git tag / github release with release notes first
# if this is prod add `-prod` suffix
VERSION=4.X.X-prod
# OR VERSION=4.X.X if dev
STAGE=prod
REGION=eu-north-1
# log in to AWS in some way first https://docs.aws.amazon.com/cli/latest/userguide/getting-started-prereqs.html#getting-started-prereqs-keys
# jq is just to extract the "Account" json-key automatically
AWS_ACCOUNT_ID=$(aws sts get-caller-identity | jq .Account)
DOCKER_REGISTRY=$AWS_ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com
TAG=$DOCKER_REGISTRY/onlineweb4-zappa:$VERSION
aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $DOCKER_REGISTRY

# If zappa is not available you must install it, alternatively use devcontainer:
poetry install -E prod
# then either run `poetry shell` first, or prepend `poetry run` before the command
zappa save-python-settings-file $STAGE

docker build . --build-arg VERSION=$VERSION -t $IMAGE --no-cache
docker push $TAG
zappa update $STAGE -d $TAG

# If you also have changed static files you must run the following:
docker build . --target=static-files -t ow4-static
ID=$(docker create ow4-static)
docker cp $ID:/srv/app/static static
BUCKET_NAME=$( yq ".${STAGE}.environment_variables.OW4_S3_BUCKET_NAME" zappa_settings.yml )
aws s3 sync static "s3://${BUCKET_NAME}/static" --delete --acl=public-read

API

Onlineweb4 comes with an API located at /api/v1.
Autogenerated Swagger-docs can be found at /api/schema/swagger-ui.