Vercel Simple Replica

This is an TypeScript/Node exercise. The goal of the project is to reproduce the basic functionality of Vercel:

1. Upload service and request handler (a router that provides the API of the application and serves the frontend),
2. Deploy service (loop worker),
3. Frontend app (built with React/Vite, by the help of Tailwind CSS and Shadcn/UI).

Technologies in use

• TypeScript
• Node
• Express
• Redis | Upstash Redis https://console.upstash.com, https://developers.cloudflare.com/workers/databases/native-integrations/upstash
• CloudFlare R2, workers, DNS, Tunnels. https://developers.cloudflare.com
• @aws-sdk v3 as client for CloudFlare R2.
• Doppler https://doppler.com
• AuthJS:
  • https://authjs.dev/getting-started/installation?framework=express
  • https://authjs.dev/reference/express
  • https://authjs.dev/reference/adapter/upstash-redis
• React/Vite
• Tailwind CSS
• Shadcn/UI
• MongoDB

References

• Code along Vercel - I built Vercel in 4 Hours (System Design, AWS, Redis, S3) by Harkirat Singh at YouTube *this is the main tutorial of this exercise
• Code along Vercel - Daily code by Harkirat Singh at his Learning Paths site
• Code along Vercel - repo by Harkirat Singh at GitHub

Daily code

Init the project

• pnpm is used for the project init and package management

mkdir -p vercel-simple-replica
cd vercel-simple-replica
git init
pnpm init
pnpm --package=typescript dlx tsc --init

• Tweak the tsconfig.json file - create the necessary directories mkdir -p dist src .
• Add few dependencies, for more details see the package.json file.
• Setup the linting and staging tools.

Coding the upload server

• See src/index.ts for the app logic.
• Code along Vercel steps - Upload service
• Coding the upload server at YouTube
• NPM: simple-git
• NPM: nodemon
• Setup nodemon and ts-node example

Node (TS) CLI hints

Run a TS file:

pnpm exec ts-node --skip-project src/utils/random.ts

Run a function from a file:

pnpm exec ts-node --skip-project -e 'require("./src/utils/random.ts").generateId()'

Setup ProcessEnv and Zod

• Validate Environment Variables With Zod article by Catalin Pit
• You're typing process.env wrong at YouTube by Matt Pocock
• DON'T USE Environment Variables Without This at YouTube by James Q Quick

Setup CloudFlare R2

• Login to CloudFlare: https://cloudflare.net/
• Go to Workers & Pages > R2 (They provides 10GB/month for free)...
  • Cloudflare API Docs
  • Cloudflare R2 Docs
• Activate the R2 option
• Create a new bucket - i.e. vercel-simple-replica
• Integrate the R2 bucket with our upload server @see the relevant section of the main tutorial
  • Copy your Account ID and create envvar CLOUDFLARE_ACCOUNT_ID,
  • Go to Manage R2 API Tokens and:
    • Create a new API Token - for example name it vercel-simple-replica-token with Permissions Object Read & Write and specify the bucket - vercel-simple-replica.
    • Create the env-vars related to the new token - CLOUDFLARE_API_TOKEN, CLOUDFLARE_API_ACCESS_KEY_ID, CLOUDFLARE_API_ACCESS_KEY_SECRET, CLOUDFLARE_API_ENDPOINT.

Migrate from aws-sdk v2 to v3

• AWS Docs: Migrate to version 3
• AWS Docs:AWS SDK for JavaScript
• Upgrading Notes (2.x to 3.x) from aws-sdk-js-v3 at GitHub

npx aws-sdk-js-codemod -t v2-to-v3 src/aws.ts

pnpm remove aws-sdk

pnpm i @aws-sdk/client-s3
pnpm i @aws-sdk/lib-storage

Put the repoId inside of a deployment que

We're using Redis as a message queue to handle the uploading process asynchronously. For this, a Redis docker container is deployed and the necessary environment variables to connect to it are defined.

To test the connection via redis-cli we can use the following command:

doppler run --command 'redis-cli -u "$REDIS_URL"'
doppler run -- sh -c 'redis-cli -u "$REDIS_URL"'

KEYS *
LPUSH build-queue test-Id-123
RPOP build-queue

• RPOP pop the first item on the right of the list/queue,
• LPOP pop the first item on the left of the list/queue.

References:

• Redis Queue > BullMQ at GitHub | BullMQ Docs
• AWS docs: What is Amazon Simple Queue Service? | Amazon Simple Queue Service

Deploy service

• Code along Vercel steps - Upload service
• Coding the upload server at YouTube

In contrast to the main tutorial, here all services are within the same TS project.

Some icons

• https://emojis.wiki/
• 💀👽🎩🤖👦💧🩸🚀🚗🔧🔩🔨🔧🪛❌💻🧲🧭🎲💾🔔🔗📥⛲🔒🧰🧊'🖥 '🧬🐾'♨️ '🌵

Build the request handler

• Code along Vercel steps - Upload service
• Build the request handler at YouTube
• Step-by-Step Guide: Uploading and Downloading Binary Files to AWS S3 v3 using serverless JavaScript

Auth.js

There are few things to do:

1. Add "type": "module" into the package.json file of your express project. Reference: @auth/express No "exports" main defined #9987
2. If you are using ts-node it will throw TypeError: Unknown file extension ".ts" with the above setup, so the easiest way is to switch to tsx. Reference: An answer from StackOverflow