Yarn doesn't properly check integrity if node_modules is cached

[glebec]

Yarn has a significant bug, as I mention on yarnpkg/yarn#2240 (comment): yarn install / yarn currently does not do a proper integrity check of the node_modules folder. If there's already a node_modules (containing a yarn.integrity), and the content of those node_modules is changed (or packages are moved between dependencies and devDependencies) without going through yarn or npm itself, then doing a yarn install will claim everything is up to date, even if that is false.

This impacts heroku-buildpack-nodejs because this buildpack caches node_modules and copies it in on deployment. If packages have moved from devDependencies to dependencies, the yarn installation step falsely claims everything is already up to date (even though there are actually missing deps), the project is built, and anything that uses those new dependencies will throw an error.

This can be mitigated in the meantime by turning off caching, meaning node_modules is not restored, which in turn means that yarn will re-generate node_modules from scratch (possibly using its own cache, which is fine and should still be more performant than npm).

heroku config:set NODE_MODULES_CACHE=false

Ideally of course, yarn will be fixed, in which case heroku-buildpack-nodejs should work fine as-is (at least as far as this issue is concerned). In the meantime:

• individual users may disable caching as mentioned above
• this buildpack might automatically disable node_modules caching for projects with a yarn.lock as a temporary workaround

---

Possibly related issues: #372, #377

[hunterloftis]

Hi @glebec thanks for the issue - we have already tried replacing node_modules caching with yarn's global cache; however, that caused such enormous build time increases for the majority of users that we reverted it.

Users who run into this issue with yarn can opt-out of caching either temporarily or permanently, but disabling it for everyone wasn't a successful option.

[glebec]

Thanks for the context @hunterloftis. Let's all hope for yarn to get fixed soon.

[glebec]

@hunterloftis — apparently there is an undocumented yarn install --check-files option which may fix this? See yarnpkg/yarn#2240 (comment).