Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rush] Build cache keys do not correctly account for configuration changes in dependencies #4400

Open
dmichon-msft opened this issue Oct 19, 2023 · 1 comment · May be fixed by #4476
Open

Comments

@dmichon-msft
Copy link
Contributor

Summary

The following options that may impact the cache key of a given operation incorrectly do not currently affect operations that depend on them:

  • dependsOnEnvVars
  • dependsOnAdditionalFiles
  • Any CLI parameter that affects some phases but does not (directly) affect others

Repro steps

Create a repository with two projects, A, and B as root folders in the repo.

Define two phases:
_phase:first. no dependencies.
_phase:second, dependency on upstream: ['_phase:first'].

Define build as a phased command with phases: ['_phase:first', '_phase:second'].

  1. Define a CLI flag parameter --production that is applicable to the command build and the phase _phase:first.

  2. Configure dependsOnEnvVars in A/config/rush-project.json for _phase:first to include TEST_VAR.

  3. Configure dependsOnAdditionalFiles in A/config/rush-project.json for _phase:first to include ../common/tmp.log.

Observe that when running rush build, the cache key for B (_phase:second) is not affected by:

  1. The presence or absence of the --production flag.
  2. Any change to the environment variable TEST_VAR.
  3. Any change to the file tmp.log

Details

The root issue here is that the cache key for an operation is computed based on the project dependencies, not the operation dependencies. Cache key for an operation should be a function of:

  1. The "configuration" of the operation, i.e. the npm script text, the CLI parameters, any configured environment variables, etc.
  2. The "local state" of the operation, i.e. the content hashes of all Git tracked files and any files identified by the dependsOnAdditionalFiles property, sorted deterministically.
  3. The final computed cache keys of all operations upon which the operation depends, sorted deterministically.
@dmichon-msft
Copy link
Contributor Author

Recommended approach to resolve this would be to store the cache key on OperationExecutionRecord and compute all keys for all operations in the graph sometime between initialization of OperationExecutionManager and beforeExecuteOperations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: High priority
Development

Successfully merging a pull request may close this issue.

1 participant