TestContext.Current.TestDirectory returns incorrect value

[yaakov-h]

In .NET Core, when inside a [TestCaseSource] property, TestContext.Current.TestDirectory returns an incorrect value.

The value it should return is the path to the directory containing the assembly being tested.

The value is does return is the path to the directory containing nunit.framework.dll inside the user's local NuGet cache.

Given the following test class:

using System;
using System.Collections.Generic;
using System.IO;
using NUnit.Framework;

namespace Tests
{
    public class Tests
    {
        [TestCaseSource(nameof(TestDirectorySource))]
        public void Test1(string testDirectory)
        {
            var assemblyDirectory = Path.GetDirectoryName(new Uri(GetType().Assembly.CodeBase).LocalPath);

            Assert.That(testDirectory, Is.EqualTo(assemblyDirectory));
        }

        public static IEnumerable<string> TestDirectorySource
        {
            get
            {
                yield return TestContext.CurrentContext.TestDirectory;
            }
        }
    }
}

When targeting .NET 4.7.2 (net472), the test passes.

When targeting .NET Core 2.1 (netcoreapp2.1), the test fails.

(Since it's a TestCaseSource, the tests also end up having different names on different platforms.)

Output of dotnet test for a multi-targeting project:

C:\Temp\NUnitDirectory>dotnet test
Build started, please wait...
Build completed.

Test run for C:\Temp\NUnitDirectory\bin\Debug\net472\NUnitDirectory.dll(.NETFramework,Version=v4.7.2)
Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 1.1367 Seconds
Test run for C:\Temp\NUnitDirectory\bin\Debug\netcoreapp2.1\NUnitDirectory.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Failed   Test1("C:\\Users\\yaakov.smith\\.nuget\\packages\\nunit\\3.10.1\\lib\\netstandard2.0")
Error Message:
   Expected string length 46 but was 69. Strings differ at index 3.
  Expected: "C:\\Temp\\NUnitDirectory\\bin\\Debug\\netcoreapp2.1"
  But was:  "C:\\Users\\yaakov.smith\\.nuget\\packages\\nunit\\3.10.1\\lib\\netsta..."
  ---------------^

Stack Trace:
   at Tests.Tests.Test1(String testDirectory) in C:\Temp\NUnitDirectory\UnitTest1.cs:line 15


Total tests: 1. Passed: 0. Failed: 1. Skipped: 0.
Test Run Failed.
Test execution time: 0.9013 Seconds

C:\Temp\NUnitDirectory>

[heshamamin]

In case no test is currently running (like in discovery step) The TestDirectory property returns the directory that contains the calling assembly. This works in the full .net framework, but in case of .net standard, the method Assembly.GetCallingAssembly() does not exist and the nunit framework assembly directory is used.
IMO TestDirectory should return null in case no test is running, but this would be a breaking change.
I normally rely on typeof(MyTestClass).GetTypeInfo().Assembly to get stable results.

[ravensorb]

Wouldn't this be a good reason to start looking at moving the eval of the TestCaseDataSource attribute to be AFTER the test case setup method?

[CharliePoole]

@ravensorb Sorry, I'm not seeing the connection you're making. Can you explain?

[ravensorb]

Sorry, I hit submit on the wrong issue. I meant to reply to one of the other open issues related to order of load with TestCase Data. Been running into a lot of issues in this area. Sorry.