Chore: upgrade jest to latest version

[alexkuc]

As a part of getting E2E functional, I am going to upgrade jest to latest version to address performance issues.

[alexkuc]

After upgrading to version 27 (current is 26) I am getting warning about multiple configs

[alexkuc]

There is a fatal error after upgrade to 27

Error log

Error: ● Invalid transformer module:
  "barista/node_modules/ts-jest/dist/index.js" specified in the "transform" object of Jest configuration
  must export a `process` or `processAsync` or `createTransformer` function.
  Code Transformation Documentation:
  https://jestjs.io/docs/code-transformation

    at barista/node_modules/@jest/transform/build/ScriptTransformer.js:387:19
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 1)
    at async ScriptTransformer.loadTransformers (barista/node_modules/@jest/transform/build/ScriptTransformer.js:379:5)
    at async createScriptTransformer (barista/node_modules/@jest/transform/build/ScriptTransformer.js:1113:3)
    at async barista/node_modules/@jest/core/build/TestScheduler.js:269:31
    at async Promise.all (index 0)
    at async TestScheduler.scheduleTests (barista/node_modules/@jest/core/build/TestScheduler.js:262:5)
    at async runJest (barista/node_modules/@jest/core/build/runJest.js:404:19)

[alexkuc]

Upgrading babel-jest and ts-jest within constraints of package.json did not help…

[alexkuc]

After quick debugging, issue with transformer points ts-jest

[alexkuc]

Upgrading ts-jest to @latest as opposed to constraints of package.json fixed error with transformer but now tests fail with ambiguous error: ReferenceError: global is not defined

[alexkuc]

Attempting to resolve it by upgrading babel-jest to @latest

[alexkuc]

The error was resolved by upgrading jest-environment-jsdom to @latest version thanks to this SO post. However, now, I am facing a new error: TypeError: Cannot read properties of undefined (reading 'testEnvironmentOptions')

[alexkuc]

⚠️ Warning ⚠️

To keep git history sane, I have rolled back (git reset --hard) to commit about fixing multiple config warnings. Then I proceeded to upgrade dependencies in the following order:

• ts-jest@latest
• bable-jest@latest
• jest-environment-jsdom@latest

[alexkuc]

Noticed this warning from ts-jest, proceeding to upgrade jest to @latest version

ts-jest[versions] (WARN) Version 27.5.1 of jest installed has not been tested with ts-jest. If you're experiencing issues, consider using a supported version (>=29.0.0 <30.0.0-0). Please do not report issues in ts-jest if you are using unsupported versions.

[alexkuc]

After upgrade, running unit tests leads to a memory leak and a system overload/halt.

[alexkuc]

Need to investigate transformer related error

Error log

    ● Invalid return value:
      `process()` or/and `processAsync()` method of code transformer found at
      "barista/config/jest/cssTransform.js"
      should return an object or a Promise resolving to an object. The object
      must have `code` property with a string of processed code.
      This error may be caused by a breaking change in Jest 28:
      https://jestjs.io/docs/28.x/upgrading-to-jest28#transformer
      Code Transformation Documentation:
      https://jestjs.io/docs/code-transformation

[alexkuc]

After fixing transformers due to breaking change in version 28.x (source), I need to address a new issue: SyntaxError: Cannot use import statement outside a module

[alexkuc]

Running a primitive test like below succeeds

it('potato', () => {
  expect(true).toBe(true);
};

[alexkuc]

Looks like yarn version 1.19+ has issues with workspaces as described here. Workaround is to use version 1.19 via yarn policies set-version 1.19.0.

[alexkuc]

Now I am facing another error: Module <rootDir>/node_modules/ts-jest in the transform option was not found.

[alexkuc]

Bizarre error gone after reinstalled node_modules

[alexkuc]

😎 update from local machine 🚀

Test Suites: 238 passed, 238 total
Tests:       1247 passed, 1247 total
Snapshots:   0 total
Time:        330.579 s
Ran all test suites.
Done in 337.46s.

⚠️ Important! isolatedModules is set to true

[alexkuc]

PR test failed due to memory exhaustion (error follows). On my local machine, the memory usage is close to 8GB whereas GitHub Action runners only have 7GB source.

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

[alexkuc]

For comparison, here's run from local machine:

Test Suites: 237 passed, 237 total
Tests:       1245 passed, 1245 total
Snapshots:   0 total
Time:        456.24 s
Ran all test suites.

⚠️ No idea where 2 tests have gone as earlier tests indicate as a total of 1247 passed tests

Edit: running yarn jest directly reveals the same results

Test Suites: 237 passed, 237 total
Tests:       1245 passed, 1245 total
Snapshots:   0 total
Time:        321.218 s, estimated 1802 s
Ran all test suites.
Done in 329.68s.

TODO: use --listTests to compare tests suites & tests with and w/o isolatedModules to find out where the 1 suite / 2 unit tests difference is coming from

[alexkuc]

Using --listTests did not reveal why I had count discrepancy. Though will skip this for now as a much bigger issue at hand is tests failing due to memory issue on GitHub Actions.

[alexkuc]

For some reason, experienced same memory related issue locally even though memory usage was <20%

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

[alexkuc]

Change branch base from FIX/e2e-tests to master

[alexkuc]

Changed testEnvironment from jest-environment-jsdom to jsdom. Also, removed babel from ts-jest config. Memory-related issues looks gone (locally) and even with 4 workers the system does not appear to crash (still running though). Although, heap size is close to 1.8~2GB per test now.

[alexkuc]

🔥 Results after running unit tests with type checking set to on 🚀

Test Suites: 236 passed, 236 total
Tests:       1249 passed, 1249 total
Snapshots:   0 total
Time:        479.306 s
Ran all test suites.
Done in 487.35s.

⚠️ Memory leak detected

A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --detectOpenHandles to find leaks. Active timers can also cause this, ensure that .unref() was called on them.

Ps. no idea why number of tests run keep varying by 2-3 units.

Edit:

To find the memory leak, I have disabled source maps in tsconfig.json and re-run tests with the command: yarn jest --verbose --logHeapUsage --detectOpenHandles. Unfortunately, I did not get a warning and did not get any indication of what could have caused the warning earlier. To double check, re-running again but this time with source maps enabled.

Edit:

After re-enabling source maps, the run has almost immediately crashed locally with the same memory error. Looks like I will have to make a factory for TS config to conditionally enable source maps since I need them to enable debugging.

[alexkuc]

Will try to narrow down where the leak happens since GitHub Actions just crashed again with the memory error

• domains/core
• domains/rem
• packages/data
• packages/dates
• packages/edtr-services
• packages/hooks
• packages/predicates
• packages/rrule-generator
• packages/services
• packages/tpc
• packages/utils

Edit: running these separately did NOT reveal where memory leak happens

[alexkuc]

It appears that memory related error is caused by --logHeapUsage

[alexkuc]

It appears memory related information is somehow related to cache but it does not explain why it happens on GitHub Actions where there is no cache in the first place

[alexkuc]

Disabling sourceMap in tsconfig.json did not make any significant performance improvements

[alexkuc]

Memory issue could be related to generation of source maps, though I am unable to establish clear cause-and-effect chain

Edit: I was able to successfully establish co-relation between source maps and jest memory-related errors locally

[alexkuc]

In terms of NodeJS version, both, my local environment and GitHub Action runner uses version 16 (source)

Edit: it appears GitHub Action runner uses version 18 (source)

Edit: do a simple echo output in runner's log revealed node version v16.20.1

[alexkuc]

So in the end it appears indeed a memory issue. By restricting memory allocation to 4GB (4096MB), I am able to successfully run all tests thanks to this comment. Documentation for this command is available here. The memory issue is known to Jest ticket, however it is still not yet resolved. Everyone's workaround is different. The issue is available here.

[alexkuc]

Locally, I am able to run the tests with the following command:

NODE_OPTIONS="--max-old-space-size=4096" yarn jest --verbose --maxWorkers=4 --maxConcurrency=2

[alexkuc]

Curios observation when running the command below

yarn jest --maxWorkers=2 --maxConcurrency=2 --forceExit

Test Suites: 236 passed, 236 total
Tests:       1249 passed, 1249 total
Snapshots:   0 total
Time:        256.018 s
Ran all test suites.
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?
Done in 262.94s.

Edit: perhaps cache from previous semi-finished run is responsible for such dramatic time reduction

[alexkuc]

Once PR #1211 is closed, this issue can be closed as well