Yarn install ignore the lock file #5270

tleveque23 · 2018-01-23T21:34:07Z

Do you want to request a feature or report a bug?

What is the current behavior?
If I read well those articles:
https://yarnpkg.com/blog/2016/11/24/lockfiles-for-all/
https://yarnpkg.com/lang/en/docs/yarn-lock/
The lock file is supposed to make sure we are all installing the same version and that a new release won't suddenly break a build. Am I right?
Well, I have a Angular 5 project that is using AngularCli to build. In our package.json file, we have:

... 
"@angular/cli": "^1.6.4",
 ...

And in the yarn.lock file:

...
"@angular/cli@^1.6.4":
  version "1.6.4"
...

Everything was working fine, then suddenly one morning, our build server reported that the build has failed! We tried to do the build on our local dev machines and everything was fine. Then I decided to delete the node_modules directory, do a yarn install and try the build.... bang! It breaks.
I then realize that the yarn.lock file has been updated. Looked inside and found that it is now using AngularCli 1.6.5! And that version has a bug that prevent our build to work.

If the current behavior is a bug, please provide the steps to reproduce.

create a new angular project with all the latest versions
In the package.json, force the AngularCli to version "1.6.4"
Do yarn install
Change the version to "^1.6.4" and make sure the lock file is still using 1.6.4
Delete the node_modules directory
Do a yarn install

What is the expected behavior?
I expect to have, in the node_modules directory, the AngularCli v.1.6.4 since it is the one in the lock file. But no, the lock file is updated, and the version present is 1.6.5.
I also tried with the options --frozen-lockfile and --pure-lockfile

Please mention your node.js, yarn and operating system version.
Node 6.9.2
Os: MacOs HightSierra

The text was updated successfully, but these errors were encountered:

rally25rs · 2018-01-29T20:09:39Z

I don't see this behavior. Mine stays at 1.6.4.

~/Projects/yarn-test 🐒   cat package.json
{
  "name": "yarn-test",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "@angular/cli": "^1.6.4"
  }
}

~/Projects/yarn-test 🐒   grep @angular/cli yarn.lock
"@angular/cli@^1.6.4":
  resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.4.tgz#422b72871776a8acca6026cbc3d21af82b197046"

~/Projects/yarn-test 🐒   git init
Initialized empty Git repository in /Users/jvalore/Projects/yarn-test/.git/

~/Projects/yarn-test (master #) 🐒   git add .

~/Projects/yarn-test (master #) 🐒   git commit -m "initial"
[master (root-commit) f8407f5] initial
 7 files changed, 5420 insertions(+)
 create mode 100644 .editorconfig
 create mode 100644 .eslintrc.json
 create mode 100644 .github/PULL_REQUEST_TEMPLATE.md
 create mode 100644 .gitignore
 create mode 100644 .travis.yml
 create mode 100644 package.json
 create mode 100644 yarn.lock

~/Projects/yarn-test (master) 🐒   rm -rf node_modules/

~/Projects/yarn-test (master) 🐒   yarn install
yarn install v1.4.0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning "@angular/cli > @angular-devkit/schematics@0.0.52" has unmet peer dependency "@angular-devkit/core@0.0.29".
warning "@angular/cli > @schematics/angular@0.1.17" has unmet peer dependency "@angular-devkit/core@0.0.29".
[4/4] 📃  Building fresh packages...
✨  Done in 8.00s.

~/Projects/yarn-test (master) 🐒   git diff

~/Projects/yarn-test (master) 🐒   grep @angular/cli yarn.lock
"@angular/cli@^1.6.4":
  resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.4.tgz#422b72871776a8acca6026cbc3d21af82b197046"

tleveque23 · 2018-01-29T21:23:12Z

First I see that you are using Yarn 1.4.0. Is that something fixed in the 1.4.x version?
Because I am using 1.3.2. The latest stable version.
And I can totally reproduce the problem. Here is my exact steps:

npm install -g @angular/cli
ng new locktest
cd locktest/
grep @angular/cli yarn.lock
"@angular/cli@1.6.6":
resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.6.tgz#002119ab2ed804bbdc86075e0095eadda2a0baa0"
vim package.json

And change the angular cli version to "1.6.4"

yarn
grep @angular/cli yarn.lock
"@angular/cli@1.6.4":
resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.4.tgz#422b72871776a8acca6026cbc3d21af82b197046"

At this point it still fine. The version 1.6.4 is in my lock file.
In fact in the lock file we have:

"@angular/cli@^1.6.4":
  version "1.6.4"
  resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.4.tgz#002119ab2ed804bbdc86075e0095eadda2a0baa0"

vim package.json

And set angularCli version to "^1.6.4"

rm -rf ./node_modules/
yarn

This is were I expect that the version in the lock file will be used... But:

grep @angular/cli yarn.lock
"@angular/cli@^1.6.4":
resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.6.tgz#002119ab2ed804bbdc86075e0095eadda2a0baa0"

yarn.lock file was updated and it now use version 1.6.6!

"@angular/cli@^1.6.4":
  version "1.6.6"
  resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.6.tgz#002119ab2ed804bbdc86075e0095eadda2a0baa0"

I really don't see the point of the yarn.lock file if it is never use.
Yes I see that in the lock file there is the ^ characters in front of the version. And it don't make sense to me to have that in the lock file. Isn't the lock file supposed to represent an exact version?
And whenever we use yarn install, the lock file is updated (if there is a new minor or patch version available).
So it seems that the only way to make sure to use the exact same version everywhere is to put exact versions in the package.json file. And that make the lock file useless even more!

The lock file should be updated only when using yarn upgrade (or created if not present of course). And maybe with a special arg with yarn install. i.e. yarn install -updatelockfile

If you think I am wrong, please, help me to understand. Give me an example where the lock file is used when we do a yarn install. Because I don't get it. And it seems that I am not alone if I google the problem....

rally25rs · 2018-01-29T21:56:37Z

I think I followed your steps correctly:

Create angular project:

~/Projects/yarn-test 🐒   ng new locktest
  create locktest/README.md (1024 bytes)
  create locktest/.angular-cli.json (1243 bytes)
  create locktest/.editorconfig (245 bytes)
  create locktest/.gitignore (529 bytes)
  create locktest/src/assets/.gitkeep (0 bytes)
  create locktest/src/environments/environment.prod.ts (51 bytes)
  create locktest/src/environments/environment.ts (387 bytes)
  create locktest/src/favicon.ico (5430 bytes)
  create locktest/src/index.html (295 bytes)
  create locktest/src/main.ts (370 bytes)
  create locktest/src/polyfills.ts (2405 bytes)
  create locktest/src/styles.css (80 bytes)
  create locktest/src/test.ts (642 bytes)
  create locktest/src/tsconfig.app.json (211 bytes)
  create locktest/src/tsconfig.spec.json (283 bytes)
  create locktest/src/typings.d.ts (104 bytes)
  create locktest/e2e/app.e2e-spec.ts (290 bytes)
  create locktest/e2e/app.po.ts (208 bytes)
  create locktest/e2e/tsconfig.e2e.json (235 bytes)
  create locktest/karma.conf.js (923 bytes)
  create locktest/package.json (1292 bytes)
  create locktest/protractor.conf.js (722 bytes)
  create locktest/tsconfig.json (363 bytes)
  create locktest/tslint.json (3012 bytes)
  create locktest/src/app/app.module.ts (316 bytes)
  create locktest/src/app/app.component.css (0 bytes)
  create locktest/src/app/app.component.html (1141 bytes)
  create locktest/src/app/app.component.spec.ts (986 bytes)
  create locktest/src/app/app.component.ts (207 bytes)
Installing packages for tooling via yarn.
yarn install v1.3.2
info No lockfile found.
[1/4] 🔍  Resolving packages...
warning karma > log4js > nodemailer@2.7.2: All versions below 4.0.1 of Nodemailer are deprecated. See https://nodemailer.com/status/
warning karma > log4js > loggly > request > node-uuid@1.4.8: Use uuid module instead
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 36.55s.
Installed packages for tooling via yarn.
Successfully initialized git.
Project 'locktest' successfully created.

~/Projects/yarn-test 🐒   cd locktest/

Manually edit package.json and set to 1.6.4:

~/Projects/yarn-test/locktest (master) 🐒   subl package.json
~/Projects/yarn-test/locktest (master) 🐒   grep angular/cli package.json
    "@angular/cli": "1.6.4",

Yarn install to update the lockfile to 1.6.4

~/Projects/yarn-test/locktest (master *) 🐒   yarn install
yarn install v1.3.2
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 4.29s.

~/Projects/yarn-test/locktest (master *) 🐒   grep angular/cli yarn.lock
"@angular/cli@1.6.4":
  resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.4.tgz#422b72871776a8acca6026cbc3d21af82b197046"

Edit package.json and yarn.lock and set both to ^1.6.4 (added the caret to both files)

~/Projects/yarn-test/locktest (master *) 🐒   grep angular/cli yarn.lock
"@angular/cli@^1.6.4":
  resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.4.tgz#422b72871776a8acca6026cbc3d21af82b197046"

~/Projects/yarn-test/locktest (master *) 🐒   grep angular/cli package.json
    "@angular/cli": "^1.6.4",

delete node_modules and rerun install

~/Projects/yarn-test/locktest (master *) 🐒   rm -rf ./node_modules/

~/Projects/yarn-test/locktest (master *) 🐒   yarn
yarn install v1.3.2
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning "@angular/cli > @angular-devkit/schematics@0.0.52" has unmet peer dependency "@angular-devkit/core@0.0.29".
warning "@angular/cli > @schematics/angular@0.1.17" has unmet peer dependency "@angular-devkit/core@0.0.29".
[4/4] 📃  Building fresh packages...
✨  Done in 17.94s.

My lockfile is unchanged:

~/Projects/yarn-test/locktest (master *) 🐒   grep angular/cli yarn.lock
"@angular/cli@^1.6.4":
  resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.4.tgz#422b72871776a8acca6026cbc3d21af82b197046"

What should happen is that Yarn will read your package.json and get the package name and version: "@angular/cli": "^1.6.4"

Then it will reformat that to {name}@{version} so "@angular/cli@^1.6.4"

Then it checks the yarn.lock for an entry with that key, which is:

"@angular/cli@^1.6.4":
  version "1.6.4"
  resolved "https://registry.yarnpkg.com/@angular/cli/-/cli-1.6.4.tgz#422b72871776a8acca6026cbc3d21af82b197046"
  dependencies:
    "@angular-devkit/build-optimizer" "~0.0.37"
    "@angular-devkit/schematics" "~0.0.44"
    "@ngtools/json-schema" "1.1.0"
    "@ngtools/webpack" "1.9.4"
    "@schematics/angular" "~0.1.12"
    autoprefixer "^7.2.3"
    chalk "~2.2.0"
    circular-dependency-plugin "^4.2.1"
    common-tags "^1.3.1"
    copy-webpack-plugin "^4.1.1"
    core-object "^3.1.0"
    css-loader "^0.28.1"
    cssnano "^3.10.0"
    denodeify "^1.2.1"
    ember-cli-string-utils "^1.0.0"
    exports-loader "^0.6.3"
    extract-text-webpack-plugin "^3.0.2"
    file-loader "^1.1.5"
    fs-extra "^4.0.0"
    glob "^7.0.3"
    html-webpack-plugin "^2.29.0"
    istanbul-instrumenter-loader "^3.0.0"
    karma-source-map-support "^1.2.0"
    less "^2.7.2"
    less-loader "^4.0.5"
    license-webpack-plugin "^1.0.0"
    loader-utils "1.1.0"
    lodash "^4.11.1"
    memory-fs "^0.4.1"
    minimatch "^3.0.4"
    node-modules-path "^1.0.0"
    nopt "^4.0.1"
    opn "~5.1.0"
    portfinder "~1.0.12"
    postcss-import "^11.0.0"
    postcss-loader "^2.0.10"
    postcss-url "^7.1.2"
    raw-loader "^0.5.1"
    resolve "^1.1.7"
    rxjs "^5.5.2"
    sass-loader "^6.0.3"
    semver "^5.1.0"
    silent-error "^1.0.0"
    source-map-loader "^0.2.0"
    source-map-support "^0.4.1"
    style-loader "^0.13.1"
    stylus "^0.54.5"
    stylus-loader "^3.0.1"
    uglifyjs-webpack-plugin "^1.1.5"
    url-loader "^0.6.2"
    webpack "~3.10.0"
    webpack-dev-middleware "~1.12.0"
    webpack-dev-server "~2.9.3"
    webpack-merge "^4.1.0"
    webpack-sources "^1.0.0"
    webpack-subresource-integrity "^1.0.1"
    zone.js "^0.8.14"
  optionalDependencies:
    node-sass "^4.3.0"

It should then use the specified version (version "1.6.4") instead of what was specified in package.json, so "1.6.4" is the exact locked version.

If the lockfile is changing even when yarn install --frozen-lockfile is run, then something is really going wrong here. Do you have a .yarnrc file somewhere (or above that project in the file system hierarchy) that might be changing configuration? Maybe another sub process or build task is executing yarn again with different arguments?

Since I can't seem to reproduce the issue, it's pretty difficult to guess at what the problem might be. If you can grab the source and help debug into it, we would appreciate it!

tleveque23 · 2018-01-30T12:49:24Z

Yes I a have a .yarnc file but it is almost empty:
lastUpdateCheck 1517257471683

To be sure it is not something on my system, I tried on a docker container. I used that image:
https://hub.docker.com/r/anapsix/alpine-java/
I choose this one just because I know it has all the tool needed like vi or nano....
Run it:
docker exec -it alpine-java bash"

Installed node.js, npm and yarn 1.3.2 on it.
And repeat the exact same process as in my last comment. And guest what? Same result. Running yarn install, the lock file is updated and now use angularCli v1.6.6!
And if I try yarn --frozen-lockfile, I get this:

bash-4.3# yarn --frozen-lockfile
yarn install v1.3.2
[1/4] Resolving packages...
error Your lockfile needs to be updated, but yarn was run with `--frozen-lockfile`.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
bash-4.3# exit
exit

Now with yarn install --pure-lockfile:

yarn install v1.3.2
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@1.1.3: The platform "linux" is incompatible with this module.
info "fsevents@1.1.3" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...

The lock file still show 1.6.4.
But if I check in the node_modules:
less node_modules/\@angular/cli/package.json
Guest what I see?:
"version": "1.6.6",

And this test with a docker container was done on my private connection at home, without any vpn connection. So out of the office.

Are you sure it is not your setup that is not right? Do you have an off-line mirror setup or something like that?

tleveque23 · 2018-01-30T17:58:12Z

I just publish my docker image if you want to try:
docker pull tleveque/yarnlocktest:firsttry

rally25rs · 2018-01-30T22:54:53Z

Docker on both my computers is all messed up unfortunately... Would it be possible to share your package.json and yarn.lock files? I'll keep trying to fix Docker on my end too. Sorry for the back-and-forth, thanks for bearing with me on this.

tleveque23 · 2018-01-31T17:25:53Z

Here is my package and lock file. Version 1.6.4 in the lock file ands ^1.6.4 defined in the package file.
packageAndLock.zip

rally25rs · 2018-01-31T17:57:58Z

Ah, alright, so here is what's going on. Your lockfile isn't in sync with your package.json file. One or the other has been edited. Specifically that version mismatch. It's basically what I explained above about how Yarn check the lockfile...

Yarn will see "@angular/cli": "^1.6.4" in package.json.
That then gets reformatted to @angular/cli@^1.6.4
Now yarn check yarn.lock for @angular/cli@^1.6.4
No entry with that key exists (@angular/cli@1.6.4 exists, but those are not the same string), so it goes to install it by checking the npm registry and getting the newest version that matches ^1.6.4

When the lockfile was created, package.json must have contained the exact version 1.6.4, then the package.json file was edited. Now the files are out of sync.

You could manually get it back in sync by changing the package.json back to an exact version, or edit the yarn.lock file and change

"@angular/cli@1.6.4":

to

"@angular/cli@^1.6.4":

tleveque23 · 2018-01-31T19:39:12Z

Ok, I understand now how it is working.
But I still think it is not the way it should work. I still think the lock file should only be updated on request (with an yarn update or with a special argument with yarn install).

rally25rs · 2018-01-31T20:13:34Z

Yeah there was a discussion in #4147 about --frozen-lockfile behavior being the default, but the behavior has never been changed.

ghost assigned torifat Jan 23, 2018

ghost added the triaged label Jan 23, 2018

rally25rs added the needs-repro-script label Jan 29, 2018

tleveque23 closed this as completed Jan 31, 2018

willdurand mentioned this issue Aug 17, 2018

use yarn.lock when building the apps mozilla/addons-frontend#5897

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Yarn install ignore the lock file #5270

Yarn install ignore the lock file #5270

tleveque23 commented Jan 23, 2018

rally25rs commented Jan 29, 2018

tleveque23 commented Jan 29, 2018

rally25rs commented Jan 29, 2018

tleveque23 commented Jan 30, 2018

tleveque23 commented Jan 30, 2018 •

edited

rally25rs commented Jan 30, 2018

tleveque23 commented Jan 31, 2018

rally25rs commented Jan 31, 2018 •

edited

tleveque23 commented Jan 31, 2018

rally25rs commented Jan 31, 2018

Yarn install ignore the lock file #5270

Yarn install ignore the lock file #5270

Comments

tleveque23 commented Jan 23, 2018

rally25rs commented Jan 29, 2018

tleveque23 commented Jan 29, 2018

rally25rs commented Jan 29, 2018

tleveque23 commented Jan 30, 2018

tleveque23 commented Jan 30, 2018 • edited

rally25rs commented Jan 30, 2018

tleveque23 commented Jan 31, 2018

rally25rs commented Jan 31, 2018 • edited

tleveque23 commented Jan 31, 2018

rally25rs commented Jan 31, 2018

tleveque23 commented Jan 30, 2018 •

edited

rally25rs commented Jan 31, 2018 •

edited