Incompetent mutants in 0.5.1 and 0.6.0

[liam-m]

Hi,

I last used mutpy in 2016 in my project https://github.com/liam-m/primes.py.

Coming back to this now, I found that the first mutant runs successfully but all subsequent ones are incompetent. This happens in 0.6.0 and 0.5.1 but not 0.4.0.

If you make the mutants individually using --mutation-number then it works as expected.

To reproduce it should be sufficient to clone the repo then run mut.py --target primes --unit-test test

I have Python 3.7.0.

[phihos]

Hi, thanks for reporting. What I found out so far: MutPy modifies a module's AST in order to create mutants. When doing that it removes the actual module by running del sys.modules[modname]. Then it loads the modified module. This behavior causes all module creations after the first one to fail with:

RuntimeError: implement_array_function method already has a docstring

NumPy seems to have issues with that behavior: numpy/numpy#14384

Now my next step is to find out, why there are no issues with version 0.4.0. I keep you updated as soon as I find out more.

[phihos]

Ok according to this issue numpy/numpy#13430 this error does not happen with NumPy versions below 1.16.0. And I just tested with with the latest MutPy + NumPy 1.15.4 and the error is gone. I try to find out more.

[phihos]

@liam-m Could you try the MutPy from the branch bugfix/fix-numpy-import-issue and tell me whether it worked for you?

[liam-m]

@phihos I've cloned the repo and checked out that branch but unfortunately I'm still seeing this issue when running bin/mut.py

[*] Mutation score [152.85278 s]: 0.0%
   - all: 1019
   - killed: 0 (0.0%)
   - survived: 1 (0.1%)
   - incompetent: 1018 (99.9%)
   - timeout: 0 (0.0%)

[phihos]

@liam-m I can not reproduce the issue with the new branch. I did the following steps:

git clone https://github.com/mutpy/mutpy.git
cd mutpy
git checkout bugfix/fix-numpy-import-issue
cd ..
git clone https://github.com/liam-m/primes.py.git
cd primes.py
virtualenv venv
source venv/bin/activate
pip install ../mutpy
pip install -r requirements.txt
mut.py --target primes --unit-test test --timeout-factor 2

The timeout-factor parameter is just for infinite loops in mutated code being aborted quicker. Some commands might be slightly different depending on your setup.
Could you try to test it again with these instructions?

[liam-m]

v0.6.1 seems to be working!