How-to have a Development-setup with docker #317

webafrancois · 2023-12-14T20:59:11Z

Which component is affected?

dev environnment

Did the issue occur at ClassQuiz.de, or on a self-hosted instance?

On ClassQuiz.de

How can the issue be reproduced?

I try to hack some css to prettify some parts of ClassQuizz (I'm not a developper, just a geek who wants to make some small contributions to this coll project), but when run "pnpm dev", I dont know how to reach the dev result.
I have clone the repo (on master branch) and I can run the docker compose wich runs fine on localhost:8000
I just want to have a docker dev env to avoid depends problems.
But if I try to run a dev service, and I have no response from Vite...

Describe the bug (with screenshots if possible)

On the repo folder, I run :
docker compose up
And I can target the localhost:8000 and run the Classquiz instanc

I can come into the docker container with :
docker exec -it classquiz-frontend-1 bash
After install curl, I can touch :

curl localhost:3000

which send :

<!--
SPDX-FileCopyrightText: 2023 Marlon W (Mawoka)

SPDX-License-Identifier: MPL-2.0
-->

<!doctype html>
<html lang="en">
	<head>
		<meta charset="utf-8" />
		<meta name="description" content="" />
		<!--		<link rel="icon" href="./favicon.png" />-->
		<meta name="viewport" content="width=device-width, initial-scale=1" />
.../...

But if I run (like I think with the manual):

pnpm dev --host 0.0.0.0

I can see :

  VITE v4.2.3  ready in 250 ms
  ➜  Local:   http://localhost:5173/
  ➜  Network: http://172.21.0.8:5173/
  ➜  press h to show help

So I guess that the Dev Vite service is running on the port 5173, like netstat shows me :

netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.11:45177        0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:5173          0.0.0.0:*               LISTEN      3128/node           
tcp        0      0 0.0.0.0:3000            0.0.0.0:*               LISTEN      19/node             
udp        0      0 127.0.0.11:35198        0.0.0.0:*                           -

But curl don't agree.

curl localhost:5173

sends nothing.

Could help me please ?

A great guy suggest me to add a second docker-compose.dev.yaml with some lines specif for dev env, but it don't work.

# SPDX-FileCopyrightText: 2023 Marlon W (Mawoka)
#
# SPDX-License-Identifier: MPL-2.0
version: "3"
services:
  frontend:
    command: "pnpm dev"
    volumes:
     - ./frontend/package.json:/app/package.json
    ports:
     - "3000:3000"

Thanks a lot.

Device

Desktop

Operating System

Linux Mint

Browser

Firefox 120

The text was updated successfully, but these errors were encountered:

mawoka-myblock · 2023-12-14T21:03:30Z

Thanks for reaching out, but it seems like you got something wrong: There's no docker-based dev-setup.

webafrancois · 2023-12-14T21:11:58Z

Thanks for the fast answer.
If I would like to have dev env, must I have only follow the guide ?

mawoka-myblock · 2023-12-14T21:12:48Z

Exactly.

webafrancois · 2023-12-14T21:24:16Z

I'm on the master branch.
Pipenv is ok (on python 3.10)
pnpm is ok (on 8.12.1 version with node v18.17.1)
But it fails on :

./run_tests.sh +

with

.../...
File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.10/site-packages/fastapi/datastructures.py", line 3, in <module>
    from starlette.datastructures import URL as URL  # noqa: F401
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.10/site-packages/starlette/datastructures.py", line 6, in <module>
    from starlette.concurrency import run_in_threadpool
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.10/site-packages/starlette/concurrency.py", line 6, in <module>
    import anyio
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.10/site-packages/anyio/__init__.py", line 27, in <module>
    from ._core._sockets import connect_tcp as connect_tcp
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.10/site-packages/anyio/_core/_sockets.py", line 33, in <module>
    from exceptiongroup import ExceptionGroup
ModuleNotFoundError: No module named 'exceptiongroup'

Should I stay on the 0.1.3 tag or anything else ?

webafrancois · 2023-12-14T21:27:31Z

I'm using Linux Mint 21.1 based on Ubuntu 22.04

mawoka-myblock · 2023-12-14T21:36:56Z

Could you try with python 3.11 again please?

webafrancois · 2023-12-14T21:45:23Z

I previously used pipenv with parameters :

pipenv --python /usr/bin/python3.10 install -d
Creating a virtualenv for this project...
Pipfile: /home/FrancoisA/projets/CEMEA/classquiz/Pipfile
Using /usr/bin/python3.10 (3.10.12) to create virtualenv...
⠴ Creating virtual environment...created virtual environment CPython3.10.12.final.0-64 in 378ms
  creator CPython3Posix(dest=/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/FrancoisA/.local/share/virtualenv)
    added seed packages: pip==23.3.1, setuptools==69.0.2, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp
Warning: Your Pipfile requires python_version 3.11, but you are using 3.10.12 (/home/F/.local/share/v/c/bin/python).
  $ pipenv --rm and rebuilding the virtual environment may resolve the issue.
  $ pipenv check will surely fail.
Installing dependencies from Pipfile.lock (94f575)...
Installing dependencies from Pipfile.lock (94f575)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

And now with python3.11 freshly installed

Creating a virtualenv for this project...
Pipfile: /home/FrancoisA/projets/CEMEA/classquiz/Pipfile
Using /usr/bin/python3.11 (3.11.0) to create virtualenv...
⠧ Creating virtual environment...created virtual environment CPython3.11.0.candidate.1-64 in 529ms
  creator CPython3Posix(dest=/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/FrancoisA/.local/share/virtualenv)
    added seed packages: pip==23.3.1, setuptools==69.0.2, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp
Installing dependencies from Pipfile.lock (94f575)...
Installing dependencies from Pipfile.lock (94f575)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

Then I run ./run_tests.sh - to stop old containers
And re-run ./run_tests.sh + which failed too because of 'charset_normalizer' :

./run_tests.sh +
mkdir: impossible de créer le répertoire «/tmp/storage»: Le fichier existe
998bc78e070a8b0b0bf801b9acdd66face0b543bcd3f2fc623414fa359340581
b9b95ec6b9e1016e7db817ea679881ca4676500c5b1fcc64e1596c011ca2e9e4
classquiz_db_data
8ec359a416c87011ad65926e5bf529c45f4ca07fd49151bfe41d0c33fefa6f8f
Traceback (most recent call last):
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/requests/compat.py", line 11, in <module>
    import chardet
ModuleNotFoundError: No module named 'chardet'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/bin/alembic", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/alembic/config.py", line 632, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/alembic/config.py", line 626, in main
    self.run_cmd(cfg, options)
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/alembic/config.py", line 603, in run_cmd
    fn(
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/alembic/command.py", line 385, in upgrade
    script.run_env()
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/alembic/script/base.py", line 582, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/FrancoisA/projets/CEMEA/classquiz/migrations/env.py", line 7, in <module>
    from classquiz.db.models import metadata
  File "/home/FrancoisA/projets/CEMEA/classquiz/classquiz/__init__.py", line 11, in <module>
    from classquiz.config import settings
  File "/home/FrancoisA/projets/CEMEA/classquiz/classquiz/config.py", line 11, in <module>
    import meilisearch as MeiliSearch
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/meilisearch/__init__.py", line 1, in <module>
    from meilisearch.client import Client as Client  # pylint: disable=useless-import-alias
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/meilisearch/client.py", line 14, in <module>
    from meilisearch._httprequests import HttpRequests
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/meilisearch/_httprequests.py", line 7, in <module>
    import requests
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/requests/__init__.py", line 45, in <module>
    from .exceptions import RequestsDependencyWarning
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/requests/exceptions.py", line 9, in <module>
    from .compat import JSONDecodeError as CompatJSONDecodeError
  File "/home/FrancoisA/.local/share/virtualenvs/classquiz-laK2BQEp/lib/python3.11/site-packages/requests/compat.py", line 13, in <module>
    import charset_normalizer as chardet
ModuleNotFoundError: No module named 'charset_normalizer'

mawoka-myblock · 2023-12-14T21:47:22Z

Try installing chardet.
https://stackoverflow.com/a/74258319

webafrancois · 2023-12-14T22:29:33Z

Maybe this issue should be closed (Because unable to install a full docker dev setup) ?
(I retry from scratch with a Booksworm Proxmox container, but problems too in ./run_tests.sh +)

webafrancois · 2023-12-15T10:58:55Z

A great guy drop me 2 files for en dev-end with docker :
Dockerfile-dev :

# SPDX-FileCopyrightText: 2023 Marlon W (Mawoka)
#
# SPDX-License-Identifier: MPL-2.0

### Build Step
# pull the Node.js Docker image
FROM node:19-bullseye-slim
# ENV API_URL=http://api:80
ENV API_URL=https://mawoka.eu
#Ah, I've noticed that!!!
ENV REDIS_URL=redis://localhost:6379
ENV VITE_MAPBOX_ACCESS_TOKEN=pk.eyJ1IjoibWF3b2thIiwiYSI6ImNsMjBob3d4ZjBhcGszYnE0bWp4aXB1ZW4ifQ.IByxV1qeIuEWpHCWsuB88A
# This Mapbox-token is restricted to the following urls: classquiz.de, classquiz.mawoka.eu, test.com
ENV VITE_HCAPTCHA=ee81b2a1-acf3-4d20-b2a4-a7ea94c7eba5
# ENV VITE_SENTRY=https://75cb4ef1be624d8f81bbaf864b722f8a@glitch.mawoka.eu/2
ENV VITE_GOOGLE_AUTH_ENABLED=true
ENV VITE_GITHUB_AUTH_ENABLED=true
ENV VITE_CAPTCHA_ENABLED=true
ENV VITE_REGISTRATION_DISABLED=True
#ENV VITE_PLAUSIBLE_DATA_URL=
# change working directory
WORKDIR /usr/src/app


# copy the package.json files from local machine to the workdir in container for dev dependencies installation
COPY package*.json ./
COPY pnpm-lock.yaml ./
COPY . .
# run npm install in our local machine
RUN corepack enable && corepack prepare pnpm@8.6.1 --activate && pnpm i
RUN pnpm i

### Serve Step

# our app is running on port 3000 within the container, so need to expose it
EXPOSE 3000

# the command that starts our app
CMD ["pnpm dev --host 0.0.0.0 --port 3000"]

And
docker-compose-dev.yml :

# SPDX-FileCopyrightText: 2023 Marlon W (Mawoka)
#
# SPDX-License-Identifier: MPL-2.0

version: "3"

services:
  frontend:
    restart: always
    build:
      context: ./frontend
      dockerfile: Dockerfile-dev
    depends_on:
      - redis
      - api
    environment:
      REDIS_URL: redis://redis:6379/0?decode_responses=True
      API_URL: http://api:80
    command: pnpm dev --host 0.0.0.0 --port 3000
    volumes:
      - ./frontend/src:/usr/src/app/src
      - ./frontend/static:/usr/src/app/static
  api:
    build: &build_cfg
      context: .
      dockerfile: Dockerfile
    restart: &restart always
    depends_on: &depends
      - db
      - redis

    environment: &env_vars
      DB_URL: "postgresql://postgres:classquiz@db:5432/classquiz"
      REDIS: "redis://redis:6379/0?decode_responses=True"
      MAIL_ADDRESS: "email@email@email.email"
      MAIL_PASSWORD: "PASSWORT"
      MAIL_USERNAME: "email@email@email.email"
      MAIL_SERVER: "email@email@email.emai"
      MAX_WORKERS: "1" # Very important
      MAIL_PORT: "587"
      SECRET_KEY: "mysecretkey"
      ACCESS_TOKEN_EXPIRE_MINUTES: 30
      SKIP_EMAIL_VERIFICATION: True
      HCAPTCHA_KEY: "HCAPTCHA_PRIVATE_KEY"
      MEILISEARCH_URL: "http://meilisearch:7700"
      STORAGE_BACKEND: "local"
      STORAGE_PATH: "/app/data"
      ROOT_ADDRESS: "http://testglorf.com:8000"
    volumes:
      - api-data:/app/data
  redis:
    image: redis:alpine
    restart: always
    healthcheck:
      test: [ "CMD", "redis-cli","ping" ]

  db:
    image: postgres:14-alpine
    restart: always
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U postgres" ]
      interval: 5s
      timeout: 5s
      retries: 5
    environment:
      POSTGRES_PASSWORD: "classquiz"
      POSTGRES_DB: "classquiz"
    volumes:
      - data:/var/lib/postgresql/data
  proxy:
    image: caddy:alpine
    restart: always
    volumes:
      - ./Caddyfile-docker:/etc/caddy/Caddyfile
    ports:
      - "8000:8080"
  meilisearch:
    image: getmeili/meilisearch:v0.28.0
    restart: always
    environment:
      MEILI_NO_ANALYTICS: true
    volumes:
      - meilisearch-data:/data.ms
  worker:
    build: *build_cfg
    environment: *env_vars
    depends_on: *depends
    restart: *restart
    command: arq classquiz.worker.WorkerSettings


volumes:
  data:
  meilisearch-data:
  api-data:

And it works with :

docker compose -f docker-compose-dev.yml up

webafrancois added the bug Something isn't working label Dec 14, 2023

mawoka-myblock added question Further information is requested needs investigation More details are needed or have to found out and removed bug Something isn't working labels Dec 14, 2023

webafrancois changed the title ~~Unable to have a Development-setup with docker~~ How-to have a Development-setup with docker Jan 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How-to have a Development-setup with docker #317

How-to have a Development-setup with docker #317

webafrancois commented Dec 14, 2023

mawoka-myblock commented Dec 14, 2023

webafrancois commented Dec 14, 2023

mawoka-myblock commented Dec 14, 2023

webafrancois commented Dec 14, 2023 •

edited

webafrancois commented Dec 14, 2023

mawoka-myblock commented Dec 14, 2023

webafrancois commented Dec 14, 2023 •

edited

mawoka-myblock commented Dec 14, 2023

webafrancois commented Dec 14, 2023 •

edited

webafrancois commented Dec 15, 2023 •

edited

How-to have a Development-setup with docker #317

How-to have a Development-setup with docker #317

Comments

webafrancois commented Dec 14, 2023

Which component is affected?

Did the issue occur at ClassQuiz.de, or on a self-hosted instance?

How can the issue be reproduced?

Describe the bug (with screenshots if possible)

Device

Operating System

Browser

mawoka-myblock commented Dec 14, 2023

webafrancois commented Dec 14, 2023

mawoka-myblock commented Dec 14, 2023

webafrancois commented Dec 14, 2023 • edited

webafrancois commented Dec 14, 2023

mawoka-myblock commented Dec 14, 2023

webafrancois commented Dec 14, 2023 • edited

mawoka-myblock commented Dec 14, 2023

webafrancois commented Dec 14, 2023 • edited

webafrancois commented Dec 15, 2023 • edited

webafrancois commented Dec 14, 2023 •

edited

webafrancois commented Dec 14, 2023 •

edited

webafrancois commented Dec 14, 2023 •

edited

webafrancois commented Dec 15, 2023 •

edited