isatty() returns false with Stryker

[regseb]

Summary

The tty.isatty(fd) method returns true when executed with Mocha. And it returns false with Stryker. I have this problem with Chalk (a library for adding color to the terminal) which doesn't colorize the text when my tests are run with Stryker. I've also reproduced the problem with picocolors.

tty.isatty(fd)

• fd <number> A numeric file descriptor
• Returns: <boolean>

The tty.isatty() method returns true if the given fd is associated with a TTY and false if it is not, including whenever fd is not a non-negative integer.

package.json

{
    "name": "testcase",
    "version": "1.0.0",
    "type": "module",
    "dependencies": {
        "@stryker-mutator/core": "7.3.0",
        "@stryker-mutator/mocha-runner": "7.3.0"
    }
}

stryker.config.js

export default {
    testRunner: "mocha",
};

test/index.js

import assert from "node:assert/strict";
import tty from "node:tty";

describe("tty", function () {
    it("isatty", function() {
        assert.ok(tty.isatty(1));
    });
});

Test runner environment

• npm install
• npx mocha

    tty
      ✔ isatty
  
  
    1 passing (2ms)
  

• npx stryker run

  1:44:39 (57824) WARN ProjectReader Warning: No files found for mutation with the given glob expressions. As a result, a dry-run will be performed without actually modifying anything. If you intended to mutate files, please check and adjust the configuration. Current glob pattern(s) used: "{src,lib}/**/!(*.+(s|S)pec|*.+(t|T)est).+(cjs|mjs|js|ts|jsx|tsx|html|vue)", "!{src,lib}/**/__tests__/**/*.+(cjs|mjs|js|ts|jsx|tsx|html|vue)". To enable file mutation, consider configuring the `mutate` property in your configuration file or using the --mutate option via the command line.
  11:44:39 (57824) INFO Instrumenter Instrumented 0 source file(s) with 0 mutant(s)
  11:44:39 (57824) INFO ConcurrencyTokenProvider Creating 7 test runner process(es).
  11:44:40 (57824) INFO DryRunExecutor Starting initial test run (mocha test runner with "perTest" coverage analysis). This may take a while.
  11:44:40 (57824) ERROR DryRunExecutor One or more tests failed in the initial test run:
          tty isatty
                  false == true
  11:44:40 (57824) ERROR Stryker There were failed tests in the initial test run.
  

Your Environment

software	version(s)
node	v21.0.0
npm	10.2.0
Operating System	Ubuntu 22.04.3 LTS

Add stryker.log

[nicojs]

Hmm, interesting. Indeed, Stryker runs your tests outside of a tty because it runs your test in a child process. You can mimic the same behavior with this script:

// @ts-check
import { exec } from "node:child_process";

exec(`node_modules/.bin/mocha test`, (error, stdout, stderr) => {
  console.log(stdout);
  console.error(stderr);
  if (error) {
    throw error;
  }
});

I'm open to suggestions on how to handle this.

[regseb]

On reflection, it's not good practice to write a test that depends on an external element. For example, redirecting the output to a file (npx mocha > report.log): the tests no longer work.

And my tests only check the case where Chalk puts color in the text. Perhaps without color, my program fails.

For my tests, I'm going to run a test with the color forced and another without.

import assert from "node:assert/strict";
import process from "node:process";
import myAwesomeProgram from "../src/my_awesome_program";

describe("myAwesomeProgram", function () {
    afterEach(function () {                                              
        delete process.env.FORCE_COLOR;                                  
    });
  
    it("support color", function() {
        process.env.FORCE_COLOR = "1";
        const result = myAwesomeProgram("foo");
        assert.equal(result, "bar");
    });

    it("support no color", function() {
        process.env.FORCE_COLOR = "0";
        const result = myAwesomeProgram("foo");
        assert.equal(result, "baz");
    });
});

---

It's still a shame that the tests behave differently between Mocha and Stryker. But the problem raised in this issue seems to come from bad tests in my project. Personally, I no longer need this difference fixed. You can close this issue if you want.