Unable to run NUnit's unit tests with ReSharper's test runner

[thomaslevesque]

I get this error:

Unit Test Runner failed to run tests

NUnit.Core.UnsupportedFrameworkException: Skipped loading assembly nunit.framework.tests because it references an unsupported version of the nunit.framework, 3.0.5831.264

Is there a way to make it work? Or is it just an unsupported scenario? Right now I have to run the tests from the command line, and when I need to debug a test, I add a Debugger.Launch at the beginning. Not very convenient...

[rprouse]

Sorry, this is just the fact that the version of Resharper you are using hasn't been updated to work with NUnit 3. When they updated to 2.6.4, they didn't update it to handle the UnsupportedFrameworkException that is thrown if it tries to run tests referencing NUnit 3.

If you want to debug your tests, you could switch your test project to execute nunit-console with the --inprocess command line so that you can just debug your tests straight from Visual Studio.

[thomaslevesque]

Sorry, this is just the fact that the version of Resharper you are using hasn't been updated to work with NUnit 3.

No, I'm using the latest stable version of ReSharper (10.0.1), which supports NUnit 3. I can run NUnit 3 tests in other projects... it's just in NUnit itself that it doesn't work.

If you want to debug your tests, you could switch your test project to execute nunit-console with the --inprocess command line so that you can just debug your tests straight from Visual Studio.

I tried that, but it remained stuck at some point and never hit my breakpoints...

[CharliePoole]

Your stacktrace is showing methods that only exist in the NUnit 2.6.4 core assembly. When the 2.6.4 core detects that you are asking it to load a 3.0 assembly, it thows that exception. Resharper, for whatever reason, is trying to run the tests using NUnit 2.6.4 assemblies.

As far as debugging, when working on NUnit you should always run tests exclusively using NUnit's own components. Otherwise, you can never be certain that the third party tool will work correctly.

In fact, it's absolutely guaranteed that Resharper doesn't work with the version of NUnit you are running because your changes have made it a newer version than any we have yet released! It's likely that resharper, if it supports NUnit 3 at all, is using release assemblies that are older versions of the same ones you are changing.

[thomaslevesque]

In fact, it's absolutely guaranteed that Resharper doesn't work with the version of NUnit you are running because your changes have made it a newer version than any we have yet released! It's likely that resharper, if it supports NUnit 3 at all, is using release assemblies that are older versions of the same ones you are changing.

OK, so this isn't a supported scenario. Thanks for the explanation.

[rprouse]

At some point you should be able to run NUnit 3 tests in Resharper, but I don't know what the current state of the Resharper runner is. The fact that it is trying to run NUnit 3 tests using NUnit 2 assemblies indicates that their runner isn't working properly.

You should report this to JetBrains.

[thomaslevesque]

At some point you should be able to run NUnit 3 tests in Resharper, but I don't know what the current state of the Resharper runner is.

R# 10.0.2 supports the final version of NUnit 3, but it's still unable to run the tests in the NUnit solution (probably for the reason explained by @CharliePoole).

I also tried to force the use of the nunit.engine.dll from the bin folder, but I get a NullReferenceException:

at NUnit.Engine.Internal.CecilExtensions.GetAttributes(TypeDefinition type, String fullName) in D:\Docs\GitHub\nunit\src\NUnitEngine\nunit.engine\Internal\CecilExtensions.cs:line 53

[CharliePoole]

Since attrType is already used successfully a few lines earlier, it seems that Resolve() is returning null. We need to check for that any place we use it. See issue #1009.

[thomaslevesque]

Since attrType is already used successfully a few lines earlier, it seems that Resolve() is returning null. We need to check for that any place we use it. See issue #1009.

Indeed, Resolve() returns null. I tried to replace this:

                attrType = attrType.Resolve().BaseType;

with this:

                var definition = attrType.Resolve();
                attrType = definition != null ? definition.BaseType : null;

Now I can run the tests, but many of them are inconclusive ("test not run"), and sometimes the runner shows another error (TargetInvocationException) that ends the test session. Even when this error doesn't occur, the test session never completes (still in status "running"), even though there doesn't seem to be any test still running. No idea if this is a bug in NUnit or in ReSharper...

[CharliePoole]

It may not be possible to successfully replace the version of the engine used by R#. Internally, we offer the option of a private version, one that the runner finds in it's own directory. Folks can specify that if they use the engine in ways that don't guarantee backward compatibility or if they merely don't want to rely on a newer release working with their runner. There's no way to tell if R# does that, however, other than asking them.

[boomer57]

Going into the ReSharper v10 Options screen, under Tools | Unit Testing | NUnit, there is an option setting for "When to use NUnit 3 test runner". Setting this to "Always" resolved the problem for me. The error message shown above no longer appears after making this change.

[Brains]

Going into the ReSharper v10 Options screen, under Tools | Unit Testing | NUnit, there is an option setting for "When to use NUnit 3 test runner". Setting this to "Always" resolved the problem for me.

Works for F# tests too. I am happy. Thanks.