Support for unified docker using S6 from LSIO

[webysther]

Closes #3111 #3093 #3092 #3091 #3069 #3060 #3036 #3009 #3007 #2999 #2990 #2581
And this PR answer multiple questions in discussions.

Description

I published this docker to provide a glympse of what can be done, hope this help to provide a better framework for the community, best whishes.

Note¹: Is necessary include pwgen and python3-pip to base docker to this work, as COPY .docker /
Note²: Here have the images ready to use: https://hub.docker.com/repository/docker/webysther/hoppscotch

Supported tags and respective Dockerfile links

hoppscotch

Hoppscotch latest and base are based on PRs #3107 and #3112 for a version unified.

Based on LSIO Images featuring:

• easy user mappings (PGID, PUID)
• custom base image with s6 overlay
• weekly base OS updates with common layers across the entire LinuxServer.io ecosystem to minimise space usage, down time and bandwidth
• regular security updates

The architecture supported by this image is:

Architecture	Available	Tag
x86-64	✅	amd64-<version tag>

Version Tags

This image provides various versions that are available via tags. Please read the descriptions carefully and exercise caution when using unstable or development tags.

Tag	Available	Description
latest	✅	Stable releases from Hoppscotch

Application Setup

• Once running the URL will be http://<host-ip>:80 and http://<host-ip>:80/admin (admin).

The application accepts a series of environment variables to further customize itself on boot.

Provided by the PRs #3107 and #3112:

Parameter	Function
-e MODE=	Default is single but can use nouser or team
-e ENABLE_ADMIN=	Default is false. Set to true to build and run admin
-e ENABLE_API=	Default is true. Build and run backend
-e ENABLE_SMTP=	Default is true. SMTP server that redirect all e-mail to stdout
-e DOMAIN=	Default is locahost.
-e SCHEMA=	Default is http. Disable httpOnly and secure from Cookie to enable working with http. We recommend change this when https when available.
-e PROXY_HOST=	Default is https://proxy.hoppscotch.io. Allow change proxy host on boot
-e PROXY_ENABLED=	Default is false. Allow change proxy enabled on boot
-e EXTENSIONS_ENABLED=	Default is false. Allow change extensions on boot
-e TELEMETRY_ENABLED=	Default is true. Allow change telemetry on boot
-e THEME_COLOR=	Default is indigo, values: green, teal, blue, indigo, purple, yellow, orange, red, pink. Allow change theme on boot
-e BG_COLOR=	Default is system, values: system, light, dark, black. Allow background color on boot
-e FONT_SIZE=	Default is small, values: small, medium, large. Allow change font size on boot
-e ZEN_MODE=	Default is false, Allow change zen mode on boot
-e VITE_ADMIN_BASE_URL=	Default is /admin. Allow start admin in a custom location
-e SMTP_PROTOCOL=	Default is smtp, values: smtp or smtps
-e SMTP_DOMAIN=	Default is localhost
-e SMTP_PORT=	Default is 25
-e SMTP_USER=	Default is nouser
-e SMTP_PASSWORD=	Default is nopass
-e POSTGRES_HOST=	Default is hoppscotch-db
-e POSTGRES_PORT=	Default is 5432
-e POSTGRES_DB=	Default is hoppscotch
-e POSTGRES_USER=	Default is postgres

Provided by hoppscotch:

Parameter	Function
-e DATABASE_URL=	Default is postgresql://postgres:testpass@hoppscotch-db:5432/hoppscotch if set the individual values for database are ignored.
-e JWT_SECRET=	Created on boot if not set. If not set by user not persist when container is recreated.
-e SESSION_SECRET=	Created on boot if not set. If not set by user not persist when container is recreated.
-e PRODUCTION=	Default is true.
-e TOKEN_SALT_COMPLEXITY=	Default is 10.
-e MAGIC_LINK_TOKEN_VALIDITY=	Default is 3.
-e REFRESH_TOKEN_VALIDITY=	Default is 604800000 (7 days).
-e ACCESS_TOKEN_VALIDITY=	Default is 86400000 (1 day).
-e REDIRECT_URL=	Default is SCHEMA+DOMAIN.
-e WHITELISTED_ORIGINS=	Default is SCHEMA+DOMAIN.
-e VITE_BASE_URL=	Default is SCHEMA+DOMAIN.
-e VITE_SHORTCODE_BASE_URL=	Default is SCHEMA+DOMAIN.
-e VITE_BACKEND_API_URL=	Default is SCHEMA+DOMAIN/api/v1.
-e VITE_BACKEND_GQL_URL=	Default is SCHEMA+DOMAIN/api/graphql.
-e VITE_BACKEND_WS_URL=	Default is wss://DOMAIN/api/graphql.
-e VITE_ADMIN_URL=	Default is SCHEMA+DOMAIN+VITE_ADMIN_BASE_URL.
-e VITE_APP_TOS_LINK=	Default is https://docs.hoppscotch.io/support/terms.
-e VITE_APP_PRIVACY_POLICY_LINK=	Default is https://docs.hoppscotch.io/support/privacy.
-e GOOGLE_CLIENT_ID=	Default is empty.
-e GOOGLE_CLIENT_SECRET=	Default is empty.
-e GOOGLE_CALLBACK_URL=	Default is SCHEMA+DOMAIN/v1/auth/google/callback.
-e GOOGLE_SCOPE=	Default is email,profile.
-e GITHUB_CLIENT_ID=	Default is empty.
-e GITHUB_CLIENT_SECRET=	Default is empty.
-e GITHUB_CALLBACK_URL=	Default is SCHEMA+DOMAIN/v1/auth/github/callback.
-e GITHUB_SCOPE=	Default is user:email.
-e MICROSOFT_CLIENT_ID=	Default is empty.
-e MICROSOFT_CLIENT_SECRET=	Default is empty.
-e MICROSOFT_CALLBACK_URL=	Default is SCHEMA+DOMAIN/v1/auth/microsoft/callback.
-e MICROSOFT_SCOPE=	Default is user.read.
-e MAILER_ADDRESS_FROM=	Default is 'SMTP_USER <SMTP_USER@SMTP_DOMAIN>'. if set the individual values for mailer are ignored.
-e MAILER_SMTP_URL=	Default is 'SMTP_PROTOCOL://SMTP_USER@DOMAIN:SMTP_PASSWORD@SMTP_DOMAIN:SMTP_PORT. if set the individual values for smtp are ignored.

Usage

Here are some example snippets to help you get started creating a container.

docker-compose (recommended, click here for more info)

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch:latest
    container_name: hoppscotch
    ports:
      - 80:80
    restart: unless-stopped

docker cli (click here for more info)

docker run -d \
  --name=hoppscotch \
  -p 80:80 \
  --restart unless-stopped \
  webysther/hoppscotch:latest

docker-compose with database

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch:latest
    container_name: hoppscotch
    networks:
      hoppscotch:
      hoppscotch-db:
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
      - POSTGRES_PASSWORD=password-for-database
    volumes:
      - /path/to/hoppscotch/config:/config
    ports:
      - 80:80
    restart: unless-stopped

  hoppscotch-db:
    image: postgres:alpine
    container_name: hoppscotch-db
    networks:
      hoppscotch-db:
    environment:
      - POSTGRES_DB=hoppscotch
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password-for-database
    volumes:
      - /path/to/postgres/data:/var/lib/postgresql/data
    expose:
      - 5432
    restart: unless-stopped

networks:
  hoppscotch:
    name: hoppscotch
  hoppscotch-db:
    name: hoppscotch-db

docker-compose for teams

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch:latest
    environment:
      - MODE=team
      - DATABASE_URL=postgresql://postgres:testpass@hoppscotch-db:5432/hoppscotch
    volumes:
      - /path/to/hoppscotch/config:/config
    ports:
      - 80:80
    restart: unless-stopped

docker-compose with UX customization

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch:latest
    environment:
      - THEME_COLOR=orange
      - BG_COLOR=dark
      - FONT_SIZE=large
      - ZEN_MODE=true
      - EXTENSIONS_ENABLED=true
      - TELEMETRY_ENABLED=false
      - DATABASE_URL=postgresql://postgres:testpass@hoppscotch-db:5432/hoppscotch
    volumes:
      - /path/to/hoppscotch/config:/config
    ports:
      - 80:80
    restart: unless-stopped

docker-compose multiple container

This configuration can be expanded to every service in container.

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch:latest
    container_name: hoppscotch
    networks:
      hoppscotch:
    environment:
      - VITE_BACKEND_API_URL=http://hoppscotch-api/api/v1
      - VITE_BACKEND_GQL_URL=http://hoppscotch-api/api/graphql
      - VITE_BACKEND_WS_URL=wss://hoppscotch-api/api/graphql
    ports:
      - 80:80
    restart: unless-stopped

  hoppscotch-api:
    image: webysther/hoppscotch:latest
    container_name: hoppscotch
    networks:
      hoppscotch:
      hoppscotch-db:
    environment:
      - POSTGRES_PASSWORD=password-for-database
    ports:
      - 8080:3170
    restart: unless-stopped

  hoppscotch-db:
    image: postgres:alpine
    container_name: hoppscotch-db
    networks:
      hoppscotch-db:
    environment:
      - POSTGRES_DB=hoppscotch
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password-for-database
    volumes:
      - /path/to/postgres/data:/var/lib/postgresql/data
    expose:
      - 5432
    restart: unless-stopped

networks:
  hoppscotch:
    name: hoppscotch
  hoppscotch-db:
    name: hoppscotch-db

Feature: MODE

MODE is ENV variable with default value of single and is used to start all services but admin.

The mode nouser is focused in use standalone without need to have a user, this mode disable SMTP Server and enable ZEN_MODE by default.

The mode team enable admin.

---
...
    environment:
      - MODE=single|nouser|team
...

Feature: Login with SMTP

By default a SMTP server spinup to provide the login facilite by e-mail direct in your docker log.
After put you e-mail the click to send a link you be logged to the docker logs, please use this link to login.
We recommend to setup a safe SMTP or OAUTH.

Feature: HTTP

By default the current distribution of hoppscotch don't allow you login using http, this is fixed in this PR and can be changed by the ENV SCHEMA.

*****************************************************
INFORMATION
Keep in mind to work in SCHEMA HTTP is necessary
disable 2 security features: httpOnly and secure
- If you is using a reverse proxy you good to go
- Don't use this option direct exposed in production!
*****************************************************

Feature: Build skipped

The current hoppscotch need to be build every time if you change any ENV values, in this PR this the build process is skipped if the ENV don't change, this will not happen if you don't set JWT_SECRET and SESSION_SECRET, because the key is recreated every time the container is restarted.

************************************************
WARNING
JWT_SECRET is automatic created but not
persist when container is recreated, set to fix.
************************************************

************************************************
WARNING
SESSION_SECRET is automatic created but not
persist when container is recreated, set to fix.
************************************************

Parameters

Container images are configured using parameters passed at runtime (such as those above). These parameters are separated by a colon and indicate <external>:<internal> respectively. For example, -p 8080:80 would expose port 80 from inside the container to be accessible from the host's IP on port 8080 outside the container.

Parameter	Function
-p 80	Allows HTTP access to the internal webserver.
-e PUID=1000	for UserID - see below for explanation
-e PGID=1000	for GroupID - see below for explanation
-e TZ=Etc/UTC	specify a timezone to use, see this list.
-v /config	Hoppscotch data

Environment variables from files (Docker secrets)

You can set any environment variable from a file by using a special prepend FILE__.

As an example:

-e FILE__PASSWORD=/run/secrets/mysecretpassword

Will set the environment variable PASSWORD based on the contents of the /run/secrets/mysecretpassword file.

User / Group Identifiers

When using volumes (-v flags) permissions issues can arise between the host OS and the container, we avoid this issue by allowing you to specify the user PUID and group PGID.

Ensure any volume directories on the host are owned by the same user you specify and any permissions issues will vanish like magic.

In this instance PUID=1000 and PGID=1000, to find yours use id user as below:

  $ id username
    uid=1000(dockeruser) gid=1000(dockergroup) groups=1000(dockergroup)

Docker Mods

LinuxServer publish various Docker Mods to enable additional functionality within the containers. The list of Mods available for this image (if any) as well as universal mods that can be applied to any one of our images can be accessed via the dynamic badges above.

Support Info

• Shell access whilst the container is running: docker exec -it hoppscotch /bin/bash
• To monitor the logs of the container in realtime: docker logs -f hoppscotch
• container version number
  • docker inspect -f '{{ index .Config.Labels "build_version" }}' hoppscotch

Updating Info

This image is static, versioned, and require an image update and container recreation to update the app inside.
Below are the instructions for updating containers:

Via Docker Compose

• Update all images: docker-compose pull
  • or update a single image: docker-compose pull hoppscotch
• Let compose update all containers as necessary: docker-compose up -d
  • or update a single container: docker-compose up -d hoppscotch
• You can also remove the old dangling images: docker image prune

Via Docker Run

• Update the image: docker pull webysther/hoppscotch:latest
• Stop the running container: docker stop hoppscotch
• Delete the container: docker rm hoppscotch
• Recreate a new container with the same docker run parameters as instructed above (if mapped correctly to a host folder, your /config folder and settings will be preserved)
• You can also remove the old dangling images: docker image prune

Via Watchtower auto-updater (only use if you don't remember the original parameters)

• Pull the latest image at its tag and replace it with the same env variables in one run:

  docker run --rm \
  -v /var/run/docker.sock:/var/run/docker.sock \
  containrrr/watchtower \
  --run-once bazarr

• You can also remove the old dangling images: docker image prune

Note: We do not endorse the use of Watchtower as a solution to automated updates of existing Docker containers. In fact we generally discourage automated updates. However, this is a useful tool for one-time manual updates of containers where you have forgotten the original parameters. In the long term, we highly recommend using Docker Compose.

Building locally

If you want to make local modifications to these images for development purposes or just to customize the logic:

git clone https://github.com/webysther/hoppscotch.git
cd hoppscotch
docker build \
  --no-cache \
  --pull \
  -t webysther/hoppscotch:latest .

Checks

• My pull request adheres to the code style of this project
• My code requires changes to the documentation
• I have updated the documentation as required
• All the tests have passed

Additional Information

[Brainpitcher]

Hi, could you please tell me why github authentication and invitations from emails lead to 401 and 404 page, everything is done by examples, but there is no way to authorize. All apps are configured properly, here is an example

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch:latest
    container_name: hoppscotch
    networks:
      hoppscotch:
      hoppscotch-db:
    environment:
      - TZ=Asia/Yekaterinburg
      - POSTGRES_PASSWORD=password-for-database
      - ENABLE_ADMIN=true
      - MODE=team
      - FONT_SIZE=small
      - SCHEMA=http
      - BG_COLOR=dark
      - TELEMETRY_ENABLED=false
#auth
      - GITHUB_CLIENT_ID=
      - GITHUB_CLIENT_SECRET=
      - GITHUB_CALLBACK_URL=http://hopp.adc-tech.ru/v1/auth/github/callback
      - GITHUB_SCOPE=user:email
#email
#      - SMTP_DOMAIN=
#      - SMTP_PORT=
#      - SMTP_USER=
#      - SMTP_PASSWORD=
      - MAILER_SMTP_URL=smtps://365test@adc-tech.ru:mypassword@mail.adc-tech.ru
      - MAILER_ADDRESS_FROM='"adc hopp" <365test@adc-tech.ru>'
#paths
      - DOMAIN=hopp.adc-tech.ru
      - VITE_BASE_URL=http://hopp.adc-tech.ru
      - VITE_SHORTCODE_BASE_URL=http://hopp.adc-tech.ru
      - VITE_ADMIN_URL=http://hopp.adc-tech.ru/admin
      - VITE_BACKEND_API_URL=http://hopp.adc-tech.ru/api/v1
      - VITE_BACKEND_GQL_URL=http://hopp.adc-tech.ru/api/graphql
      - VITE_BACKEND_WS_URL=wss://hopp.adc-tech.ru/api/graphql
    volumes:
      - /home/sdo/hopp/config:/config
    ports:
      - 80:80
    restart: unless-stopped

  hoppscotch-db:
    image: postgres:alpine
    container_name: hoppscotch-db
    networks:
      hoppscotch-db:
    environment:
      - POSTGRES_DB=hoppscotch
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password-for-database
    volumes:
      - /home/sdo/hopp/data:/var/lib/postgresql/data
    expose:
      - 5432
    restart: unless-stopped

networks:
  hoppscotch:
    name: hoppscotch
  hoppscotch-db:
    name: hoppscotch-db

[webysther]

Hi @Brainpitcher, try remove the smtp configuration and look the logs if the link works, if not, put here the output log after "Custom build". If works maybe you need to check the port of smtp, if you use smtps is expected 465, smtp is 25.

Tip: You dont need to set VITE_* anymore, all URL's use the domain as a base, try remove and look the docker logs.

Try this, if works the mail, the problem is the SMTP configuration, this sample works exaclty like your, I removed the configuration not need anymore.

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch
    container_name: hoppscotch
    networks:
      hoppscotch:
      hoppscotch-db:
    environment:
      - TZ=Asia/Yekaterinburg
      - POSTGRES_PASSWORD=password-for-database
      - MODE=team
      - BG_COLOR=dark
      - TELEMETRY_ENABLED=false
      - GITHUB_CLIENT_ID= #auth
      - GITHUB_CLIENT_SECRET= #auth
#paths
      - DOMAIN=hopp.adc-tech.ru
    volumes:
      - /home/sdo/hopp/config:/config
    ports:
      - 80:80
    restart: unless-stopped

  hoppscotch-db:
    image: postgres:alpine
    container_name: hoppscotch-db
    networks:
      hoppscotch-db:
    environment:
      - POSTGRES_DB=hoppscotch
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password-for-database
    volumes:
      - /home/sdo/hopp/data:/var/lib/postgresql/data
    expose:
      - 5432
    restart: unless-stopped

networks:
  hoppscotch:
    name: hoppscotch
  hoppscotch-db:
    name: hoppscotch-db

After confirm the SMTP is works, try this one:

PS: Sometimes the smtp is trick.

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch
    container_name: hoppscotch
    networks:
      hoppscotch:
      hoppscotch-db:
    environment:
      - TZ=Asia/Yekaterinburg
      - POSTGRES_PASSWORD=password-for-database
      - DOMAIN=hopp.adc-tech.ru
      - MODE=team
      - BG_COLOR=dark
      - TELEMETRY_ENABLED=false
      - GITHUB_CLIENT_ID= #auth
      - GITHUB_CLIENT_SECRET= #auth
      - SMTP_DOMAIN=mail.adc-tech.ru
      - SMTP_PORT= #port (check using telnet)
      - SMTP_USER=adc-tech.ru
      - SMTP_PASSWORD=mypassword
    volumes:
      - /home/sdo/hopp/config:/config
    ports:
      - 80:80
    restart: unless-stopped

  hoppscotch-db:
    image: postgres:alpine
    container_name: hoppscotch-db
    networks:
      hoppscotch-db:
    environment:
      - POSTGRES_DB=hoppscotch
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password-for-database
    volumes:
      - /home/sdo/hopp/data:/var/lib/postgresql/data
    expose:
      - 5432
    restart: unless-stopped

networks:
  hoppscotch:
    name: hoppscotch
  hoppscotch-db:
    name: hoppscotch-db

[Brainpitcher]

Hi @Brainpitcher, try remove the smtp configuration and look the logs if the link works, if not, put here the output log after "Custom build". If works maybe you need to check the port of smtp, if you use smtps is expected 465, smtp is 25.

i have tried to do this the link from logs still leads to 401, оr to 404 if i try to reach it from another browser

Hi @Brainpitcher, try remove the smtp configuration and look the logs if the link works, if not, put here the output log after "Custom build". If works maybe you need to check the port of smtp, if you use smtps is expected 465, smtp is 25.

Tip: You dont need to set VITE_* anymore, all URL's use the domain as a base, try remove and look the docker logs.

Try this, if works the mail, the problem is the SMTP configuration, this sample works exaclty like your, I removed the configuration not need anymore.

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch
    container_name: hoppscotch
    networks:
      hoppscotch:
      hoppscotch-db:
    environment:
      - TZ=Asia/Yekaterinburg
      - POSTGRES_PASSWORD=password-for-database
      - MODE=team
      - BG_COLOR=dark
      - TELEMETRY_ENABLED=false
      - GITHUB_CLIENT_ID= #auth
      - GITHUB_CLIENT_SECRET= #auth
#paths
      - DOMAIN=hopp.adc-tech.ru
    volumes:
      - /home/sdo/hopp/config:/config
    ports:
      - 80:80
    restart: unless-stopped

  hoppscotch-db:
    image: postgres:alpine
    container_name: hoppscotch-db
    networks:
      hoppscotch-db:
    environment:
      - POSTGRES_DB=hoppscotch
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password-for-database
    volumes:
      - /home/sdo/hopp/data:/var/lib/postgresql/data
    expose:
      - 5432
    restart: unless-stopped

networks:
  hoppscotch:
    name: hoppscotch
  hoppscotch-db:
    name: hoppscotch-db

After confirm the SMTP is works, try this one:

PS: Sometimes the smtp is trick.

---
version: "2.1"
services:
  hoppscotch:
    image: webysther/hoppscotch
    container_name: hoppscotch
    networks:
      hoppscotch:
      hoppscotch-db:
    environment:
      - TZ=Asia/Yekaterinburg
      - POSTGRES_PASSWORD=password-for-database
      - DOMAIN=hopp.adc-tech.ru
      - MODE=team
      - BG_COLOR=dark
      - TELEMETRY_ENABLED=false
      - GITHUB_CLIENT_ID= #auth
      - GITHUB_CLIENT_SECRET= #auth
      - SMTP_DOMAIN=mail.adc-tech.ru
      - SMTP_PORT= #port (check using telnet)
      - SMTP_USER=adc-tech.ru
      - SMTP_PASSWORD=mypassword
    volumes:
      - /home/sdo/hopp/config:/config
    ports:
      - 80:80
    restart: unless-stopped

  hoppscotch-db:
    image: postgres:alpine
    container_name: hoppscotch-db
    networks:
      hoppscotch-db:
    environment:
      - POSTGRES_DB=hoppscotch
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password-for-database
    volumes:
      - /home/sdo/hopp/data:/var/lib/postgresql/data
    expose:
      - 5432
    restart: unless-stopped

networks:
  hoppscotch:
    name: hoppscotch
  hoppscotch-db:
    name: hoppscotch-db

I removed vite stuff and now everything works like a charm, except github auth, but i suppose it's because of http schema. BTW have you managed to make it work?
Really appreciate your help! You are doing so much for this awesome project!
I wish i could by you a beer but all i can do it is rate you a star)

P/S emails works just fine with just only two env

      - MAILER_SMTP_URL=smtps://365test@adc-tech.ru:password@mail.adc-tech.ru
      - MAILER_ADDRESS_FROM='"adc hopp" <365test@adc-tech.ru>'

[webysther]

@Brainpitcher I'm not the maintainer, but thanks anyway! I have a hard time to get this project working, so I image this can help others. So, about github, I think only by https because reqs from github, but http is always a temporary option, right? Mostly use to test before put in production with https using letsencrypt, etc...

If you want to help with money try sponsor button in the project! 👍🏻

[webysther]

Hey folks! How works the order/priorization for PR review? I saw only organization members is enable to review and a mode 'benevolent dictator' behavior and more recent PR from these members get reviewed. I miss something?

[AndrewBastin]

@webysther we are going through your PR, there are some sizeable changes with the setup here and we are playing around to see how everything works.

Your work is indeed brilliant, but please let us have our time to go through it and understand it :)

[webysther]

Hey @AndrewBastin, thanks for the feedback, I don't receive any notice, thats why I asked, take your time, thanks!

[RaHehl]

Hi,

i try to deploy the webysther/hoppscotch:latest image into k8s via helm.

I'm getting the error "s6-chown: fatal: unable to chown /run: Operation not permited"

I found this https://docs.k8s-at-home.com/our-container-images/permissions/ "With Kubernetes, s6-overlay is not needed."

I'm not very deep into the topic, do you have any ideas on how I could solve the problem?

[Brainpitcher]

Hi,

i try to deploy the webysther/hoppscotch:latest image into k8s via helm.

I'm getting the error "s6-chown: fatal: unable to chown /run: Operation not permited"

I found this https://docs.k8s-at-home.com/our-container-images/permissions/ "With Kubernetes, s6-overlay is not needed."

I'm not very deep into the topic, do you have any ideas on how I could solve the problem?

i am trying to do the same thins now, can you tell me how did you manage to pass envs inside the container and what chart did you use?

[RaHehl]

Hi,
i try to deploy the webysther/hoppscotch:latest image into k8s via helm.
I'm getting the error "s6-chown: fatal: unable to chown /run: Operation not permited"
I found this https://docs.k8s-at-home.com/our-container-images/permissions/ "With Kubernetes, s6-overlay is not needed."
I'm not very deep into the topic, do you have any ideas on how I could solve the problem?

i am trying to do the same thins now, can you tell me how did you manage to pass envs inside the container and what chart did you use?

I tried a internal helm example which is a slight modified version of this example https://github.com/helm/examples where i replaced the nginx image with webysther/hoppscotch:latest .

at the current step i havn't added vars

[webysther]

Hi @rhehl indeed, why you need s6 all-in-docker if you will use k8s? With k8s is expected you have resources to run every service in a pod, I suppose. I think this PR #3107 address this problem.

My suggestion is search how run LSIO images on k8s, but I don't find no problems related to use s6 with k8s, they state just you dont need.

In you case, set PUID and GUID this maybe be enough.

[webysther]

A PR of a helm file you be really nice for the project @Brainpitcher! I don't use k8s for my homelab...

[webysther]

My suggestion if use the default example I provided and try kompose.

[Brainpitcher]

A PR of a helm file you be really nice for the project @Brainpitcher! I don't use k8s for my homelab...

Yeap, i am just working to do my best) I have nо choice because of my boss :) i am close

[Brainpitcher]

My suggestion if use the default example I provided and try kompose.

nope, i think we should use basic helm and make some templates, it almost works, just need to win-win with envs and pvc's. BTW is it possible to get code of the Dockerfile?

[webysther]

I don't get it, what you mean by "get the code of the Dockerfile?"?

[Brainpitcher]

I don't get it, what you mean by "get the code of the Dockerfile?"?

i mean the original Dockerfile that was used during the image build process. i was unable to find it in your repository and i decided to aks you as an author of image image for the Dockerfile.

[webysther]

Okay, I get-it

• Copy from the Added base docker image #3107, or
• Use this one with the mod that I said in the initial paragraph of this PR:

# syntax=docker/dockerfile:1

FROM lsiobase/ubuntu:jammy

ARG HOPPSCOTH_RELEASE
ARG HOPPSCOTH_REPO="hoppscotch/hoppscotch"
LABEL build_version="Based on Linuxserver.io images version:- ${HOPPSCOTH_RELEASE}"
LABEL maintainer="Hoppscotch"

ARG DEBIAN_FRONTEND="noninteractive"

#deps - nvm, node, npm, pnpm
ENV NODE_VERSION=18.16.0
ENV PATH="/root/.nvm/versions/node/v${NODE_VERSION}/bin/:${PATH}"
ENV NVM_DIR=/root/.nvm
RUN echo "**** install node family ****" && \
    apt update && apt install -y debian-keyring debian-archive-keyring apt-transport-https pwgen python3-pip && \
    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash  && \
    . "$NVM_DIR/nvm.sh" && nvm install ${NODE_VERSION} && \
    nvm use v${NODE_VERSION} && nvm alias default v${NODE_VERSION} && \
    npm install -g npm pnpm && node --version && npm --version && pnpm --version

#deps - caddy
RUN echo "**** install caddy ****" && \
    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | \
    gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg && \
    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | \
    tee /etc/apt/sources.list.d/caddy-stable.list && \
    apt update && \
    apt install -y caddy && \
    caddy version

RUN echo "**** install hoppscoth ****" && \
  mkdir -p \
    /app/ && \
  if [ -z ${HOPPSCOTH_RELEASE+x} ]; then \
    HOPPSCOTH_RELEASE=$(curl -sX GET "https://api.github.com/repos/${HOPPSCOTH_REPO}/releases/latest" \
    | awk '/tag_name/{print $4;exit}' FS='[""]'); \
  fi && \
  mkdir -p /app && \
  echo $HOPPSCOTH_RELEASE && \
  curl -o \
    /tmp/hoppscotch.tar.gz -L \
    "https://github.com/${HOPPSCOTH_REPO}/archive/${HOPPSCOTH_RELEASE}.tar.gz" && \
  tar xf \
    /tmp/hoppscotch.tar.gz -C \
    /app/ --strip-components=1

WORKDIR /app
RUN echo "**** install app ****" && \
      cp .env.example .env && \
      pnpm install

RUN echo "**** cleanup ****" && \
  apt-get clean && \
  rm -rf \
    /tmp/* \
    /var/lib/apt/lists/* \
    /var/tmp/*

COPY .docker /

VOLUME /config

• Merge this PR or just copy the .docker folder into the original repository
• Run:

docker build --build-arg HOPPSCOTH_RELEASE="2023.4.5" . -t webysther/hoppscotch:2023.4.5

With build-arg you have multiple ways to customize, now you have the same image!

[webysther]

docker build --build-arg HOPPSCOTH_RELEASE="main" . -t webysther/hoppscotch:main
docker build --build-arg HOPPSCOTH_RELEASE="e3dd9e99a1951f795149b70bcba46295cf09f712" . -t webysther/hoppscotch:e3dd9e9

[webysther]

A note: using this image is needed only 2GB of RAM, I used to test a VM with dynamic memory allocation and no SWAP. For run the image the history is another, at least ~16GB, but in this case I really recommend to use 16GB of SWAP and 2GB RAM, the daily usage is around this value, keep swap high is important to avoid change the configuration in every image update. The build with 16RAM took around 3 minutes, with SWAP (in NVMve), around 5 minutes.

[AndrewBastin]

I am grouping this PR with #3107 and the remarks here apply there as well.

So, we have gone through reviewing this PR and the changes it brings.

And we have decided that we are not going to be bringing this into the official repo for the following set of reasons:

• We are not a fan of how the script given here modifies the frontend hoppscotch-common code to update the default frontend settings values. This ties the script to code and how it looks like, we think a better solution is to be able to define an environment variable that can override default configs or something of that sort. The implementation provided here is mixing up config with code in a very non-formalized way.
• We do acknowledge that this PR tackles the ability to easily compose as a single convenient container and we do know the community really wants such an implementation and we hear ya! We want one to. We just ran into some difficulties containerizing the frontend projects (selfhost-web and sh-admin), we have a solution in the works now which we hope to share with the people in the coming weeks, with that we will be able to prebuild the whole container so you can just pull and use it without more build steps. Since that is in the works, we feel this implementation doesn't help out at the moment.
• We have had issues running this container in arm64. Although, I do feel like this can be a trivial fix. We have a commitment we have established in the past to support ARM.

I am not going to lie though, @webysther this implementation and your effort in trying to fix all of the things is amazing and appreciated. I was not even that familiar with LSIO and S6 Overlay and over the past weeks going through your implementation and all the nuances I am starting to appreciate a lot of aspects of it. If you reach out in our Discord server (or maybe even DM me on LinkedIn), I would definitely love to talk to you and setup a more formal relationship, because we would definitely love your expertise in making Hoppscotch better for everyone!

So in the meantime, community members can use webysther/hoppscotch docker image to support this use case, just know that we are most probably going to have an official implementation eventually.

Also to add, Hoppscotch Self Hosting is still beta (as mentioned in https://docs.hoppscotch.io). This does not represent how self hosted will look like when it stabilizes or our quality of work and commitment to the self hosting (we are super for it), we are working with the community to try to make it a fully functioning thing that works for everyone and we request the community's patience until everything falls into place.

[varunpan]

I have used this to setup my selfhosted hoppscotch and it works amazing and really simple to setup. The only issue I have is that Google Oauth does not work and take me to a 404 page not found within hoppscotch.

I am using nginx reverse proxy (LSIO Swag container) for this to have HTTPS for security.
My compose-file:

version: "3.7"
networks:
  br0:
    name: br0
    external: true
services:
  hoppscotch:
    image: webysther/hoppscotch:latest
    container_name: hoppscotch
    labels:
      com.centurylinklabs.watchtower.enable: "false"
    environment:
      - MODE=team
      - DATABASE_URL=${DATABASE_URL}
      - ENABLE_ADMIN=true
      - ENABLE_API=true
      - DOMAIN=hoppscotch.domain.com
      - SCHEMA=https
      - GOOGLE_CLIENT_ID=${GOOGLE_CLIENT_ID}
      - GOOGLE_CLIENT_SECRET=${GOOGLE_CLIENT_SECRET}
      - MAILER_SMTP_URL=${MAILER_SMTP_URL}
      - MAILER_ADDRESS_FROM=${MAILER_ADDRESS_FROM}
      - TIMEZONE=${TZ}
      - PUID=${PUID}
      - PGID=${PGID}
    ports:
      - 3010:80
    restart: unless-stopped
    networks:
      br0:
        ipv4_address: 172.18.5.219

My env variables:

DATABASE_URL=postgresql://hoppscotchadmin:password@192.168.1.5:5432/hoppscotch
MAILER_SMTP_URL=smtp://email@domain.com:pass@smtp.gmail.com
MAILER_ADDRESS_FROM=info <info@simplementix.com>
GOOGLE_CLIENT_ID=<redacted>
GOOGLE_CLIENT_SECRET=<redacted>
TZ=America/New_York
PUID=1000
PGID=1000

My nginx proxy conf:

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    server_name hoppscotch.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_app 192.168.1.4;
        set $upstream_port 3010;
        set $upstream_proto http;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;
    }
}

Google OAuth Cred setup:

Authorized JavaScript origins: https://hoppscotch.domain.com
Authorized redirect URIs: NONE

Im not sure what I am missing to get google OAuth working? Also is there a way to restrict usage of google OAuth to specific domain?

[webysther]

Hi @varunpan the problem seems tobe the nginx configuration, the code need to have a catch all solution, without this the router for PWA don't work. I will put here my nginx configuration:

server {
  set $forward_scheme http;
  set $server         "hoppscotch";
  set $port           80;

  listen 80;
  listen [::]:80;

  listen 443 ssl http2;
  listen [::]:443 ssl http2;

  server_name postman.webysther.org;

  # Let's Encrypt SSL
  include conf.d/include/letsencrypt-acme-challenge.conf;
  include conf.d/include/ssl-ciphers.conf;
  ssl_certificate /etc/letsencrypt/live/npm-57/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/npm-57/privkey.pem;

  # Block Exploits
  include conf.d/include/block-exploits.conf;

  # HSTS (ngx_http_headers_module is required) (63072000 seconds = 2 years)
  add_header Strict-Transport-Security "max-age=63072000;includeSubDomains; preload" always;

  # Force SSL
  include conf.d/include/force-ssl.conf;

  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection $http_connection;
  proxy_http_version 1.1;

  location / {
    include /data/nginx/custom/default/hsts.conf;
    include /data/nginx/custom/default/websocket.conf;

    proxy_pass $forward_scheme://$server:$port;

    auth_request /authelia;
    auth_request_set $target_url $scheme://$http_host$request_uri;

    auth_request_set $user $upstream_http_remote_user;
    auth_request_set $groups $upstream_http_remote_groups;
    auth_request_set $name $upstream_http_remote_name;
    auth_request_set $email $upstream_http_remote_email;

    proxy_set_header Remote-User $user;
    proxy_set_header Remote-Groups $groups;
    proxy_set_header Remote-Name $name;
    proxy_set_header Remote-Email $email;

    client_body_buffer_size 128k;

    proxy_next_upstream error timeout invalid_header http_500 http_502 http_503;

    send_timeout 5m;
    proxy_read_timeout 360;
    proxy_send_timeout 360;
    proxy_connect_timeout 360;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Forwarded-Uri $request_uri;
    proxy_set_header X-Forwarded-Ssl on;
    proxy_redirect  http://  $scheme://;
    proxy_set_header Connection "";
    proxy_cache_bypass $cookie_session;
    proxy_no_cache $cookie_session;
    proxy_buffers 64 256k;

    set_real_ip_from 10.0.0.0/8;
    set_real_ip_from 172.0.0.0/8;
    set_real_ip_from 192.168.0.0/16;
    set_real_ip_from fc00::/7;
    real_ip_header X-Forwarded-For;
    real_ip_recursive on;
  }
}

This is nginx-proxy-manager+authelia+customs, but look around the code and issues, I just change the configuration before build and if is not related to this build probally is more related to the source code.

If you look the code about login have lots of problems Dependency inversion principle and I don't wast my time trying to configure to myself, I recommend to stick to email until code become less instable.

[NicklasWallgren]

So in the meantime, community members can use webysther/hoppscotch docker image to support this use case, just know that we are most probably going to have an official implementation eventually.

Has there been any progress in regards to the official implementation? I've been out of the loop for a while, so I might have missed something.