New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: unified dockerfile #1840
Merged
Merged
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
1c7ed24
feat: unified dockerfile
bodinsamuel 0ee94b4
Merge branch 'master' into feat/unified-dockerfile
bodinsamuel 8e4e242
working
bodinsamuel 34146d5
test
bodinsamuel 0143bbc
more
bodinsamuel 200b921
working
bodinsamuel 7b3f08c
push
bodinsamuel 9683845
do not wait
bodinsamuel ffd9252
error to null
bodinsamuel af42935
why is everything so hard
bodinsamuel 607b40a
let's try
bodinsamuel a53ceaa
build less
bodinsamuel 50ceed1
fail push
bodinsamuel 209904c
test with everything
bodinsamuel f2b1d85
remove from dockerfile
bodinsamuel 5e9c800
use matrix
bodinsamuel 8bc5598
remove concurrency
bodinsamuel 35c2b57
try reusable
bodinsamuel 2bb699f
a
bodinsamuel cbfecb8
missing secret
bodinsamuel c129ec2
fix condition
bodinsamuel c74b6df
fix
bodinsamuel 532ec92
lock
bodinsamuel 6397bae
Merge branch 'master' into feat/unified-dockerfile
bodinsamuel d0f7ed2
more
bodinsamuel fa755dd
rename
bodinsamuel ce8b7ba
-__
bodinsamuel 1a89229
test
bodinsamuel 24d0b26
revert to something working
bodinsamuel 9be501f
do not compile other
bodinsamuel 72189e8
clean up dockerfile
bodinsamuel 2d3d3e3
Merge branch 'master' into feat/unified-dockerfile
bodinsamuel 16764b3
Merge branch 'master' into feat/unified-dockerfile
bodinsamuel a06d05c
review
bodinsamuel 7bc7354
review
bodinsamuel File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
name: Build unified Docker image | ||
|
||
on: | ||
workflow_call: | ||
inputs: | ||
if: | ||
description: 'Whether to run this job' | ||
required: false | ||
default: true | ||
type: boolean | ||
name: | ||
required: true | ||
type: string | ||
key_for_sentry_secret: | ||
required: false | ||
type: string | ||
key_for_posthog_secret: | ||
required: false | ||
type: string | ||
|
||
jobs: | ||
build-container: | ||
if: ${{ inputs.if }} | ||
|
||
runs-on: ubuntu-latest | ||
env: | ||
CAN_PUSH: "${{ secrets.DOCKER_PASSWORD != ' && secrets.DOCKER_USERNAME != ' }}" | ||
SHA: ${{ github.event.pull_request.head.sha || github.sha }} | ||
|
||
steps: | ||
- name: Check out | ||
uses: actions/checkout@v4 | ||
|
||
- name: Set up QEMU | ||
uses: docker/setup-qemu-action@v3 | ||
|
||
- name: Set up Docker Buildx | ||
uses: docker/setup-buildx-action@v3 | ||
|
||
- name: Log in to Docker Hub | ||
uses: docker/login-action@v3 | ||
if: env.CAN_PUSH == 'true' | ||
with: | ||
username: '${{ secrets.DOCKER_USERNAME }}' | ||
password: '${{ secrets.DOCKER_PASSWORD }}' | ||
|
||
# Needed for buildx gha cache to work | ||
- name: Expose GitHub Runtime | ||
uses: crazy-max/ghaction-github-runtime@v2 | ||
|
||
- name: Build image (${{ inputs.name }}) | ||
run: | | ||
export SENTRY_KEY=${{ secrets[inputs.key_for_sentry_secret] }} | ||
export POSTHOG_KEY=${{ secrets[inputs.key_for_posthog_secret] }} | ||
./scripts/build_docker.sh build ${{ inputs.name }} ${{ env.SHA }} | ||
|
||
- name: Push image | ||
if: env.CAN_PUSH == 'true' | ||
run: | | ||
docker push nangohq/nango:${{ inputs.name }}-${{ env.SHA }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
name: '[Release] Build unified Docker image' | ||
|
||
on: | ||
push: | ||
branches: | ||
- master | ||
- staging/** | ||
pull_request: | ||
|
||
jobs: | ||
build-image: | ||
strategy: | ||
matrix: | ||
group: | ||
- name: 'staging' | ||
if: true | ||
sentry_key: SENTRY_KEY_staging | ||
|
||
- name: 'prod' | ||
if: ${{ github.ref == 'refs/heads/master' }} | ||
sentry_key: SENTRY_KEY_prod | ||
posthog_key: POSTHOG_KEY_prod | ||
|
||
# Commented for now | ||
# - name: 'enterprise' | ||
# if: ${{ github.ref == 'refs/heads/master' }} | ||
|
||
secrets: inherit | ||
uses: ./.github/workflows/build-image-reusable.yaml | ||
with: | ||
if: ${{ matrix.group.if }} | ||
name: ${{ matrix.group.name }} | ||
key_for_sentry_secret: ${{ matrix.group.sentry_key }} | ||
key_for_posthog_secret: ${{ matrix.group.posthog_key }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
# ------------------ | ||
# New tmp image | ||
# ------------------ | ||
FROM node:18.19.1-bullseye-slim AS build | ||
|
||
# Setup the app WORKDIR | ||
WORKDIR /app/tmp | ||
|
||
# Copy and install dependencies separately from the app's code | ||
# To leverage Docker's cache when no dependency has changed | ||
COPY packages/frontend/package.json ./packages/frontend/package.json | ||
COPY packages/jobs/package.json ./packages/jobs/package.json | ||
COPY packages/node-client/package.json ./packages/node-client/package.json | ||
COPY packages/persist/package.json ./packages/persist/package.json | ||
COPY packages/runner/package.json ./packages/runner/package.json | ||
COPY packages/server/package.json ./packages/server/package.json | ||
COPY packages/shared/package.json ./packages/shared/package.json | ||
COPY packages/webapp/package.json ./packages/webapp/package.json | ||
COPY package*.json ./ | ||
|
||
# Install every dependencies | ||
RUN true \ | ||
&& npm i | ||
|
||
# At this stage we copy back all sources | ||
COPY . /app/tmp | ||
|
||
# Build the backend separately because it can be cached even when we change the env vars | ||
RUN true \ | ||
&& npm run ts-build:docker | ||
|
||
# /!\ Do not set NODE_ENV=production before building, it will break some modules | ||
# ENV NODE_ENV=production | ||
ARG image_env | ||
ARG posthog_key | ||
ARG sentry_key | ||
|
||
# TODO: remove the need for this | ||
ENV REACT_APP_ENV $image_env | ||
ENV REACT_APP_PUBLIC_POSTHOG_HOST https://app.posthog.com | ||
ENV REACT_APP_PUBLIC_POSTHOG_KEY $posthog_key | ||
ENV REACT_APP_PUBLIC_SENTRY_KEY $sentry_key | ||
|
||
# Build the frontend | ||
RUN true \ | ||
&& npm run -w @nangohq/webapp build | ||
|
||
# Clean src | ||
RUN true \ | ||
&& rm -rf packages/*/src \ | ||
&& rm -rf packages/*/lib \ | ||
&& rm -rf packages/webapp/public \ | ||
&& rm -rf packages/webapp/node_modules | ||
|
||
# Clean dev dependencies | ||
RUN true \ | ||
&& npm prune --omit=dev --omit=peer --omit=optional | ||
|
||
# ---- Web ---- | ||
# Resulting new, minimal image | ||
FROM node:18.19.1-bullseye-slim as web | ||
|
||
|
||
# - Bash is just to be able to log inside the image and have a decent shell | ||
RUN true \ | ||
&& apt update && apt-get install -y bash ca-certificates \ | ||
&& update-ca-certificates \ | ||
&& rm -rf /var/lib/apt/lists/* \ | ||
&& apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false | ||
|
||
# Do not use root to run the app | ||
USER node | ||
|
||
WORKDIR /app/nango | ||
|
||
# Code | ||
COPY --from=build --chown=node:node /app/tmp /app/nango | ||
|
||
ARG image_env | ||
ARG git_hash | ||
|
||
ENV PORT=8080 | ||
ENV NODE_ENV=production | ||
ENV IMAGE_ENV $image_env | ||
ENV GIT_HASH $git_hash | ||
|
||
EXPOSE 8080 | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -e | ||
|
||
ACTION=$1 | ||
ENV=$2 # enterprise | hosted | prod | staging | ||
GIT_HASH=$3 | ||
|
||
USAGE="./build_docker.sh <build|push> <enterprise | hosted | prod | staging> GIT_HASH" | ||
RED='\033[0;31m' | ||
YELLOW='\033[0;33m' | ||
NC='\033[0m' | ||
|
||
if [ "$ACTION" != "push" ] && [ "$ACTION" != "build" ]; then | ||
echo -e "${RED}Please specify an action${NC}\n" | ||
echo "$USAGE" | ||
exit | ||
fi | ||
|
||
if [ "$ENV" != "enterprise" ] && [ "$ENV" != "hosted" ] && [ "$ENV" != "prod" ] && [ "$ENV" != "staging" ]; then | ||
echo -e "${RED}Please specify an environment${NC}\n" | ||
echo "$USAGE" | ||
exit | ||
fi | ||
|
||
if [ -z $GIT_HASH ]; then | ||
echo -e "${RED}GIT_HASH is empty${NC}" | ||
exit | ||
fi | ||
|
||
if [ -z $SENTRY_KEY ]; then | ||
echo -e "${YELLOW}SENTRY_KEY is empty${NC}" | ||
fi | ||
if [ -z $POSTHOG_KEY ]; then | ||
echo -e "${YELLOW}POSTHOG_KEY is empty${NC}" | ||
fi | ||
|
||
# Move to here no matter where the file was executed | ||
cd "$(dirname "$0")" | ||
|
||
tags="-t nangohq/nango:${ENV}-${GIT_HASH}" | ||
|
||
if [ $ACTION == 'build' ]; then | ||
tags+=" --output=type=docker" | ||
else | ||
tags+=" --output=type=registry" | ||
fi | ||
|
||
echo "" | ||
echo -e "Building nangohq/nango:$ENV\n" | ||
|
||
docker buildx build \ | ||
--platform linux/amd64 \ | ||
--build-arg image_env="$ENV" \ | ||
--build-arg git_hash="$GIT_HASH" \ | ||
--build-arg posthog_key="$SENTRY_KEY" \ | ||
--build-arg sentry_key="$POSTHOG_KEY" \ | ||
--cache-from type=gha \ | ||
--cache-to type=gha,mode=max \ | ||
--file ../Dockerfile \ | ||
$tags \ | ||
../ | ||
bodinsamuel marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
{ | ||
"files": [], | ||
"references": [ | ||
{ | ||
"path": "packages/frontend" | ||
}, | ||
{ | ||
"path": "packages/jobs" | ||
}, | ||
{ | ||
"path": "packages/node-client" | ||
}, | ||
{ | ||
"path": "packages/persist" | ||
}, | ||
{ | ||
"path": "packages/runner" | ||
}, | ||
{ | ||
"path": "packages/server" | ||
}, | ||
{ | ||
"path": "packages/shared" | ||
}, | ||
{ | ||
"path": "packages/webapp" | ||
} | ||
] | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we currently use port 80. That would require an infra change if yes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
locally we use 300x
on render we pass port in an env var, e.g:
NANGO_JOBS_PORT
I don't think it will be much more complicated than changing the env var but my test on staging will tell us