yarn.build

yarn.BUILD is a plugin for Yarn 4 (berry). It uses your dependency graph to build just whats needed, when it's needed. You can setup a monorepo with a few backend packages, a server package, maybe a graphQL schema package, and a frontend package. And build it all, in the order it's needed. Then, only rebuild when something changes.

See the full docs at yarn.BUILD

To install for Yarn 4:

yarn plugin import https://yarn.build/latest

Or install any of the commands individually with

yarn plugin import https://yarn.build/latest/build
yarn plugin import https://yarn.build/latest/test
yarn plugin import https://yarn.build/latest/bundle

If you're upgrading the plugin the install location has changed to be under the @yarn.build namespace. If you have any yarn.build plugin previously installed you may need to remove the old one manually from .yarnrc.yml:

plugins:
  - checksum: ...                                 <-- remove this entry if both exist
    path: .yarn/plugins/@ojkelly/plugin-all.cjs   <--  
    spec: 'https://yarn.build/latest'             <--
  - checksum: ...
    path: .yarn/plugins/@yarn.build/plugin-all.cjs
    spec: 'https://yarn.build/latest'

OpenTelemetry Support

yarn.build's build, test and bundle commands now come with optional OpenTelemetry (OTEL) instrumentation.

To use it, you need to run an OTEL Collector with a http receiver:

receivers:
  otlp:
    protocols:
      grpc:
      http: # this is the one we need, it defaults to port 4318

And set the appropirate envar for example OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 if you are running the collector on the same host as you're running yarn.build.

NOTE: yarn.build doesn't currently support the grpc endpoint, becuase bundling the required .proto files might need a rework of the yarn plugin bundler, which is out of scope of the intial yarn.build OTEL integration.

Commands

build

Build your package and all dependencies.

Run in the root of your project, or in a non-workspace folder to build everything.

Run in a specific workspace to build that workspace and all of its dependencies in the correct order, only rebuidling what's changed.

My builds are never cached?

Yarn build tries to guess your input and output folders based on common conventions.

If they're different you can specify them explicitly in package.json:

  "yarn.build": {
    "input": "src",
    "output": "dist"
  }

If that still doesn't work, check to see if your build script is modifying any files in your input folder. Some build tools like to mess with files like tsconfig.json and others.

The most ideal state is that your input folder is never modified by your build step. If this continues to happen, you should try to adjust the build scripts, or workspace layout to avoid it.

As this is fundemental to ensuring sound builds, yarn build will never cache the input folder if it's changed.

Exclude

Pass --exclude or --exclude-current to selectively exclude packages from being built.

Pass -v for verbose to get a print out of which packages were skipped or excluded.

yarn build --exclude packages/example/lorem-ipsum
yarn build --exclude packages/example/*

# Globs for package names work too, but you need to quote them so your shell doesn't try to substitute it
yarn build --exclude "@internal*"


# For Dev
# this one is really useful at the start of a dev command or similar where you
# are watching for changes in the current workspace but need to ensure your
# dependencies are built
yarn build --exclude-current

NOTE: if you explicitly exclude a workspace that another workspace depends on, and that workspace is being built the command may fail.

Git / CI integration

Use the flag --changes, to ignore the build cache, and build everything with changes staged or in the last commit.

Use --since-branch main to ignore the build cache, and build everything with changes based on what git says is different between the current branch and main (or another branch of your choosing).

Use --since ${COMMIT_HASH} to ignore the build cache, and build everything with changes between the current commit and the provided one.

query

Run yarn build query from within a package to see the dependency graph of what might be built.

Query doesn't currently show what's cached / needs to be rebuilt.

bundle

Bundle a package and its local dependencies, designed for containers and AWS lambda.

A file entrypoint.js is added to the project root, that reexports the file you specify as main in package.json.

Output bundle.zip to a specific folder

# or any path you want to put it in
yarn bundle --output-directory ../tmp

Bundle but don't zip

This is useful when you're building inside a docker container.

Choose an output directory outside your project and pass --no-compress.

# or any path you want to put it in that's outside your project root
yarn bundle --no-compress --output-directory /srv/app

See this Dockerfile and build script for an example of how you can bundle into a container image.

.bundleignore

You can set files to be ignored when bundling for even smaller bundles.

Add a .bundleignore file with the same format as .gitignore next to the package.json you are bundling.

Optionally put one next to your root package.json to apply to all bundles.

You can pass --ignore-file to specify a different ignore file.

Or decide at bundle time what to ignore by passing --exclude along with the file path to ignore.

See #112 for the original PR.

test

Test your package and it's dependencies.

Config

By default yarn.build looks at your package.json and chooses some reasonable defaults.

{
  "name": "@internal/lorem-ipsum",
  "version": "1.0.0",
  "main": "build/index.js",
  "license": "UNLICENSED",
  "private": true,
  "scripts": {
    "build": "tsc",
    "test": "jest"
  }
}

When you specify main yarn.build will exclude that folder from the build tracker, and use the package root (the same directory as the package.json) as the input folder to track.

If you want to customise the input and output folders per package you can setup a package.json as follows:

{
  "name": "@internal/lorem-ipsum",
  "version": "1.0.0",
  "license": "UNLICENSED",
  "private": true,
  "scripts": {
    "build": "tsc -outDir dist",
    "test": "jest"
  },
  "yarn.build": {
    "input": ".",
    "output": "dist"
  }
}

Troubleshooting

**The output is interlaced, or mangled, or not useful in CI**

yarn.build uses is-ci to check if it's running in a CI environment, and will not print progress in the same way it does when run locally (or with an interactive tty).

Typically is-ci is really good at detecting a CI environment. It does this by checking a for one of many known environment variables set by CI tools. Including the most common and most useful fallback CI=true.

If you run yarn build or yarn test wrapped inside another execution environment inside your CI pipeline, you might need to pass an environment variable (ENV) to let yarn.build know it's being run in CI.

Depending on how your script is run, you can do something like the following:

CI=true yarn build

Adapted for Docker / BuildKit, the following will set CI for the script, but not the whole container. See issue #5 for more information

RUN env CI=true yarn build

---

plugin-package-yaml

Have you ever wanted to write you package.json as package.yaml or even package.yml?

Well now you can!

To install:

yarn plugin import https://yarn.build/yaml

How to use `plugin-package-yaml`

Once installed, any folder with a package.yaml and without a package.json will run through this plugin. This lets you opt-in packages that don't have any tooling that requires package.json to be present on disk.

Swap an existing package.json over to a package.yaml by converting it's contents to YAML, and renaming the file.

This plugin will transparently convert your package.yaml back into json for all of Yarn's tooling, meaning Yarn has no idea it's not writing to a package.json.

name: "@internal/lorem-ipsum"
version: 1.0.0
main: dist/index.js

# license, none for the example
license: UNLICENSED
private: true

# scripts comment
scripts:
  build: tsc
  test: jest
  dev: ts-node ./src/index.ts

dependencies:
  "@internal/phrase-lorem-ipsum": "workspace:*"
  jest: "^26"
  ts-jest: "^26.4.4"
  typescript: ^4.3.5

devDependencies:
  "@types/node": ^16.4.1
  ts-node: ^10.1.0
  "@types/jest": ^26.0.24

jest:
  preset: ts-jest

# here we define our input and output
# as we defined main above, we don't need this
# if your output directory is different or not easily definable in main
# specify it here
yarn.build:
  input: .
  output: dist

Caveats

Existing tooling that wants to read from your package.json will break, unless it reads it via Yarn.

Troubleshooting

If it breaks, convert your yaml package file back to json, and comment the plugin out from .yarnrc.yml.

Please also make an issue describing you problem, so we can hopefully fix it.

Example

The initial usecase for this is for non-javascript packages in a polyglot yarn.build repository. As an example this is how you can build a go app, leveraging yarn and yarn.build but with a yaml file as your build specification (ie package.yaml).

In this example we have a graphql schema defined in typescript that generates type files we can consume in our go binary.

name: "@internal/server"
version: 1.0.0
main: cmd/main.go

# license, none for the example
license: UNLICENSED
private: true

# scripts comment
scripts:
  build: GOOS=linux GOARCH=amd64 go build -o .build/main cmd/main.go
  test: go test ./...
  dev: go run cmd/main.go

dependencies:
  "@internal/graphql-schema": "workspace:*"

yarn.build:
  input: .
  output: .build

---

For developing on this repository see packages/plugins/readme.md