Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bun Jest runner incorrect timeouts #11147

Closed
stefreak opened this issue May 17, 2024 · 1 comment
Closed

Bun Jest runner incorrect timeouts #11147

stefreak opened this issue May 17, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@stefreak
Copy link

What version of Bun is running?

1.1.8+89d25807f

What platform is your computer?

MacOS

What steps can reproduce the bug?

Run bun test with the following contents in foo.test.js:

// foo.test.js
import { afterEach, beforeAll, beforeEach, describe, test } from "bun:test";

describe("timeout bugs", async () => {
  beforeEach(async () => {
    await new Promise((resolve) => setTimeout(resolve, 200));
  });
  afterEach(async () => {
    await new Promise((resolve) => setTimeout(resolve, 200));
  });
  test(
    "first test",
    async () => {
      await new Promise((resolve) => setTimeout(resolve, 200));
    },
    {
      timeout: 220,
    }
  );
  test(
    "second test",
    async () => {
      await new Promise((resolve) => setTimeout(resolve, 600));
    },
    {
      timeout: 1000,
    }
  );
  test(
    "third test",
    async () => {
      await new Promise((resolve) => setTimeout(resolve, 1000));
    },
    {
      timeout: 1300,
    }
  );
  test(
    "fourth test",
    async () => {
      await new Promise((resolve) => setTimeout(resolve, 1000));
    },
    {
      timeout: 0,
    }
  );
  test(
    "the only test that should actually time out",
    async () => {
      await new Promise((resolve) => setTimeout(resolve, 1000));
    },
    {
      timeout: 100,
    }
  );
});

What is the expected behavior?

All tests, except for the last test, should succeed:

foo.test.js:
✓ timeout bugs > first test [203.10ms]
✓ timeout bugs > second test [601.91ms]
✓ timeout bugs > third test [1002.49ms]
✓ timeout bugs > fourth test [1004.94ms]
error: Test "the only test that should actually time out" timed out after 100ms
✗ timeout bugs > the only test that should actually time out [103.41ms]

 4 pass
 1 fail

What do you see instead?

% bun test
bun test v1.1.8 (89d25807)

foo.test.js:
✓ timeout bugs > first test [203.33ms]
error: Test "second test" timed out after 1000ms
✗ timeout bugs > second test [442.58ms]
✓ timeout bugs > third test [1002.92ms]
error: Test "fourth test" timed out after 0ms
✗ timeout bugs > fourth test [1302.32ms]
error: Test "the only test that should actually time out" timed out after 100ms
✗ timeout bugs > the only test that should actually time out [103.78ms]

 2 pass
 3 fail
Ran 5 tests across 1 files. [3.07s]

Additional information

I have a fix and am about to create a PR for this issue

@stefreak stefreak added the bug Something isn't working label May 17, 2024
stefreak added a commit to stefreak/bun that referenced this issue May 17, 2024
Fixes oven-sh#11147

Jest runner would report incorrect timeouts, due to races in the timer
accounting code.

This commit arms and disarms timers before and after every test to
ensure correct timeouts without false-positives.
stefreak added a commit to stefreak/bun that referenced this issue May 17, 2024
Fixes oven-sh#11147

Jest runner would report incorrect timeouts, due to races in the timer
accounting code.

This commit arms and disarms timers before and after every test to
ensure correct timeouts without false-positives.
@stefreak stefreak changed the title Bun Jest runner timeout false-negatives and false-positives Bun Jest runner timeout false-positives May 17, 2024
@stefreak stefreak changed the title Bun Jest runner timeout false-positives Bun Jest runner incorrect timeouts May 17, 2024
@stefreak
Copy link
Author

Fixed by #11419 – the test suite passes now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant