[BUG] check-module-boundaries.js sometimes fails: EPERM: operation not permitted, rename 'project-graph.json~ab440dc1' -> 'project-graph.json'

[brnbs]

Current Behavior

We use Jenkins for automated builds (with Windows build agents) and sometimes encounter the following error during the build process. Upon restarting, the error typically disappears, suggesting that the issue might be due to a race condition.

Error message from console:

> nx run ghos-spoofing-engine:build--force-debug

Executing Command: dotnet "--version"
8.0.204
Executing Command: dotnet "build" "W:\my_project_root\packages\ghos-spoofing-engine\ghos-spoofing-engine.csproj" "--configuration" "Debug" "--no-dependencies" "--verbosity" "minimal"
MSBuild version 17.9.8+b34f75857 for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
  ghos-spoofing-engine -> W:\my_project_root\dist\packages\ghos-spoofing-engine\Ghos.External.dll
  ERROR (1) when writing 
  EPERM: operation not permitted, rename 'W:\my_project_root\.nx\cache\project-graph.json~ab440dc1' -> 'W:\my_project_root\.nx\cache\project-graph.json'
EXEC : error : EPERM: operation not permitted, rename 'W:\my_project_root\.nx\cache\project-graph.json~ab440dc1' -> 'W:\my_project_root\.nx\cache\project-graph.json' [W:\my_project_root\packages\ghos-spoofing-engine\ghos-spoofing-engine.csproj]
      at renameSync (node:fs:1040:3)
      at writeCache (W:\my_project_root\node_modules\nx\src\project-graph\nx-deps-cache.js:102:39)
      at buildProjectGraphUsingProjectFileMap (W:\my_project_root\node_modules\nx\src\project-graph\build-project-graph.js:71:40)
      at async buildProjectGraphAndSourceMapsWithoutDaemon (W:\my_project_root\node_modules\nx\src\project-graph\project-graph.js:78:27)
      at async createProjectGraphAndSourceMapsAsync (W:\my_project_root\node_modules\nx\src\project-graph\project-graph.js:136:25)
      at async createProjectGraphAsync (W:\my_project_root\node_modules\nx\src\project-graph\project-graph.js:125:39)
  Checking module boundaries for ghos-spoofing-engine
  CONFIGURED undefined
W:\my_project_root\Directory.Build.targets(11,5): error MSB3073: The command "node ..\..\/node_modules/@nx-dotnet/core/src/tasks/check-module-boundaries.js --project-root "packages\ghos-spoofing-engine"" exited with code -1. [W:\my_project_root\packages\ghos-spoofing-engine\ghos-spoofing-engine.csproj]

Build FAILED.

EXEC : error : EPERM: operation not permitted, rename 'W:\my_project_root\.nx\cache\project-graph.json~ab440dc1' -> 'W:\my_project_root\.nx\cache\project-graph.json' [W:\my_project_root\packages\ghos-spoofing-engine\ghos-spoofing-engine.csproj]
W:\my_project_root\Directory.Build.targets(11,5): error MSB3073: The command "node ..\..\/node_modules/@nx-dotnet/core/src/tasks/check-module-boundaries.js --project-root "packages\ghos-spoofing-engine"" exited with code -1. [W:\my_project_root\packages\ghos-spoofing-engine\ghos-spoofing-engine.csproj]
    0 Warning(s)
    2 Error(s)

Time Elapsed 00:00:16.96

 �[31m>�[39m �[0m�[7m�[1m�[31m NX �[39m�[22m�[27m�[0m  �[31mdotnet execution returned status code 1�[39m

Expected Behavior

The check-module-boundaries.js script should execute successfully without any errors, like this:

MSBuild version 17.9.8+b34f75857 for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
  ghos-spoofing-engine -> W:\my_project_root\dist\packages\ghos-spoofing-engine\Ghos.External.dll
  Checking module boundaries for ghos-spoofing-engine
  CONFIGURED undefined

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.86

Github Repo

No response

Steps to Reproduce

I cannot reproduce this issue consistently, but it occurs in approximately 30% of our automated builds.

Nx Report

Node   : 18.15.0
   OS     : win32-x64
   npm    : 9.5.0
   
   nx                 : 17.2.8
   @nx/js             : 17.2.8
   @nx/jest           : 17.2.8
   @nx/linter         : 17.2.8
   @nx/eslint         : 17.2.8
   @nx/workspace      : 17.2.8
   @nx/angular        : 17.2.8
   @nx/cypress        : 17.2.8
   @nx/devkit         : 17.2.8
   @nx/esbuild        : 17.2.8
   @nx/eslint-plugin  : 17.2.8
   @nx/node           : 17.2.8
   @nrwl/tao          : 17.2.8
   @nx/web            : 17.2.8
   @nx/webpack        : 17.2.8
   typescript         : 4.8.4
   ---------------------------------------
   Community plugins:
   @nx-dotnet/core : 2.1.2
   ngx-bootstrap   : 6.1.0
   nx-electron     : 17.0.1

nx.json

{
  "pluginsConfig": {
    "@nx/js": {
      "analyzeSourceFiles": true
    }
  },
  "extends": "nx/presets/npm.json",
  "$schema": "./node_modules/nx/schemas/nx-schema.json",
  "affected": {
    "defaultBase": "master"
  },
  "targetDefaults": {
    "build": {
      "cache": true,
      "inputs": ["default"],
      "dependsOn": [
        {
          "dependencies": true,
          "target": "build",
          "params": "forward"
        }
      ]
    },
    "publish": {
      "dependsOn": [
        {
          "dependencies": true,
          "target": "publish",
          "params": "forward"
        }
      ]
    },
    "assemble": {
      "cache": true,
      "inputs": ["default"],
      "dependsOn": [
        {
          "dependencies": true,
          "target": "assemble",
          "params": "forward"
        }
      ]
    },
    "lint": {
      "cache": true,
      "inputs": ["default"]
    },
    "test": {
      "cache": true,
      "inputs": ["default"]
    }
  },
  "namedInputs": {
    "default": ["{projectRoot}/**/*", "!{projectRoot}/**/*.md"]
  },
  "plugins": ["@nx-dotnet/core"]
}

Failure Logs

No response

Additional Information

I believe the issue comes from this specific call:

nx-dotnet/packages/core/src/tasks/check-module-boundaries.ts

Line 179 in a8561e2

const graph = await createProjectGraphAsync();

The initial command in the Jenkinsfile is npx nx affected --base=develop -t test --configuration=production --exclude=ghos-browser-tests --parallel=5 that leads to run the failing nx run ghos-spoofing-engine:build--force-debug command.

[brnbs]

Today got the same issue in the same repository but with another package and file name (file-map.json):

MSBuild version 17.9.8+b34f75857 for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
  ghos-cli-domain -> W:\my_project\dist\packages\ghos-cli-domain\ghos-cli-domain.dll
  ERROR (1) when writing 
  EPERM: operation not permitted, rename 'W:\my_project\.nx\cache\file-map.json~ee6cefc6' -> 'W:\my_project\.nx\cache\file-map.json'
EXEC : error : EPERM: operation not permitted, rename 'W:\my_project\.nx\cache\file-map.json~ee6cefc6' -> 'W:\my_project\.nx\cache\file-map.json' [W:\my_project\packages\ghos-cli-domain\ghos-cli-domain.csproj]
      at renameSync (node:fs:1040:3)
      at writeCache (W:\my_project\node_modules\nx\src\project-graph\nx-deps-cache.js:104:39)
      at buildProjectGraphUsingProjectFileMap (W:\my_project\node_modules\nx\src\project-graph\build-project-graph.js:71:40)
      at async buildProjectGraphAndSourceMapsWithoutDaemon (W:\my_project\node_modules\nx\src\project-graph\project-graph.js:78:27)
      at async createProjectGraphAndSourceMapsAsync (W:\my_project\node_modules\nx\src\project-graph\project-graph.js:136:25)
      at async createProjectGraphAsync (W:\my_project\node_modules\nx\src\project-graph\project-graph.js:125:39)
  Checking module boundaries for ghos-cli-domain
  CONFIGURED undefined
W:\my_project\Directory.Build.targets(11,5): error MSB3073: The command "node ..\..\/node_modules/@nx-dotnet/core/src/tasks/check-module-boundaries.js --project-root "packages\ghos-cli-domain"" exited with code -1. [W:\my_project\packages\ghos-cli-domain\ghos-cli-domain.csproj]