Switch to Yarn?

[filiptronicek]

While installing dependencies for Open Sauced, I noticed how long the dependencies are installing. On a beefy VM in the cloud (64GB RAM, Intel Xeon (16 threads) @ 2.8 GHz), it took a whopping 2 minutes and 17 seconds to install everything.

On the same machine, it only took Yarn to do the same thing just 35 seconds! That's almost 4x faster!

Isn't Yarn something to at least think about with these massive improvements?

[filiptronicek]

WDYT about this, @bdougie?

[bdougie]

I would be curious what the benchmark is with npm7. I think the yarn team is always to innovate faster, but I have a conflict of interest since I work closely with the npm team (I am interested in leveraging npm and providing feedback directly).

Once this PR is ready we can benchmark: #1017

[0-vortex]

My machine so far, open sauced, random old Gatsby and a freshly baked CRA:

• yarn, no quotes JSON is ~3 seconds faster
• pnpm, YAML is ~5 seconds faster

[mtfoley]

FWIW: https://jeromewu.github.io/how-to-speed-up-node-js-modules-installation/

[0-vortex]

From a security perspective strongly disagree with all points there. The idea that removing the loglevel would increase performance by 3x is quite entertaining and only relevant depending on how big the dependency tree is and resolution algorithm used to process them. Quite a lot of diversity here - https://github.com/jeromewu/npm-vs-yarn-in-cicd/tree/master/cra-app 💯

The verdaccio instance does not seem to be configured past defaults either, its only purpose as far as I can tell to speed up 304 registry requests and providing same DNS, at that point it seems like purely networking decisions and cpu I/O limitations :<

[0-vortex]

Should be a personal development choice whether one wants to use npm or yarn - the main motivation for doing this is:

1. don't care about servers, they are running predictable builds that can be optimized with minimal effort separately than our developer experience
2. if one uses any other tool than the preferred "project choice" the cache would not exist or be outdated, it would download ~500MB of data in chunks and waste roughly 3GB of expanded hard drive space

Benchmark wise things are pretty simple, shrinkwrap dependencies with npm and yarn will issue warning npm-shrinkwrap.json found. This will not be updated or respected. See https://yarnpkg.com/en/docs/migrating-from-npm for more information. (choice); similarly, npm will not know how to deal with a "custom" lockfile like yarn.lock - the trick here is to bake the shrinkwrap command into package-lock.json and then run yarn on top of it - if deep resolution tree that is on your own and might require you to run npm install and yarn install on top of a hot node_modules folder. Doing that makes the actual yarn optimization evident and basically what happens is npm doesn't remove quotes from JSON - if yarn tries to install from package-lock.json it will take longer to parse the JSON output and both npm ci and yarn install will essentially be equivalent - that is achieved and true because running install on another manager just changes dependency resolution in v2 lockfile enabled projects.

time npm ci
rm -rf node_modules
time yarn install --frozen-lockfile

🙄

Optimized dependencies we are comparing 10 seconds to 13 seconds and similar timers, yarn brings some opinionated choices at the expense documentation, would like to have them both but then the docs fall on us to create or yarn users themselves 🚀

TL;DR: should be able to use whatever modern package manager and it will just work

[0-vortex]

@filiptronicek I think the project is ready to clone and run using any package manager now. Not looking to stray away from npm but having a good standard configuration, we should be able to document the minor quirks of using the other package managers - contributing dependency updates or new packages should follow the contributing docs and get tested with npm.

Have gotten a clean macOS 12 install since my last comment, am available for benchmarking and recording for npm, yarn and pnpm if you want to maybe pair on this 🔥