As a workaround, it is possible to use the Range attribute on the test parameter(s):

[Test, AutoData]
public void Test(
    [System.ComponentModel.DataAnnotations.Range(-1, 1)]
    int actual)
{
    Assert.True(-1 <= actual && actual <= 1); // Passes.
}

That seems to work, but only gets run once with -1. So System.ComponentModel.DataAnnotations.Range and NUnit doesn't work nicely together.

It doesn't work with NUnit's RangeAttribute.

You can also use Enums in Values, which you can't in Range, e.g:

public enum TestData
{
    One = 1,
    Two = 2,
    Three = 3,
    Four = 4
}

[Test, AutoData]
public void Test_Enum(
    string name,
    [Values(TestData.Two, TestData.Three)]
    TestData testData)
{
    Assert.IsNotNullOrEmpty(name);
    Assert.True(testData == TestData.Two || testData == TestData.Three);
}

Another thing I have noticed is that AutoData doesn't pick a random enum, it always picks the first one.

Regarding Range, you should try with RandomRangedNumberCustomization:

I spent some time investigating this issue. From the NUnit docs

By default, NUnit creates test cases from all possible combinations of the datapoints provided on parameters - the combinatorial approach.

    MyTest(1, "A")
    MyTest(1, "B")
    MyTest(2, "A")
    MyTest(2, "B")
    MyTest(3, "A")
    MyTest(3, "B")
[Test]
public void MyTest(
    [Values(1,2,3)] int x,
    [Values("A","B")] string s)
{
    ...
}

That means tha AutoFixture's AutoDataAttribute instead of returning a flat list of specimens (x and y) must return all the possible combinations of the Values as it's shown in the sample above.

There is a draft fix for the original issue that makes the test pass. But, I noticed that even in master all the Scenario tests may work or fail depending of the test runner! That is definitely another issue, and if somebody wants to continue this work - you're welcome ;)

Hey!

I'm looking a bit at this issue.

I got a few questions:

Is there a reason for having both AutoDataAttribute and InlineAutoData? At least form API point of view, AutoDataAttribute covers just a special case of InlineAutoDataAttribute, i.e. no values provided.

Should the NUnit attributes be supported by both AutoData attributes? In any case, is it ok that the same attribute returns a single test under certain conditions, many tests under others?

Also, there is a naming issues. AutoFixture uses System.ComponentModel.DataAnnotations attributes whilst NUnit uses its own breed of attributes (found in NUnit.Framework). In some cases, like the RangeAttribute, we might have a naming overlap with the NUnit returning all the values in the range and AutoFixture generating a value within the range.

Finally, how much of this is only affecting the NUnit glue library?

Is there a reason for having both AutoDataAttribute and InlineAutoData? At least form API point of view, AutoDataAttribute covers just a special case of InlineAutoDataAttribute, i.e. no values provided.

In fact yes, the difference is in the semantic meaning only. I believe it originated from xUnit, where we have InlineData attribute.
While the API might be indeed simplified to a single attribute only, I'd prefer to keep that as is. First of all because a lot of people became are familiar with both attributes. Another reason is that I see value in that distinction - in semantics of the test.

I'm not sure whether those attributes are natural to NUnit, as I'm not using this test framework. However, for xUnit it's completely fine.

As for the second part of your question, I think that we might want to revisit the whole approach. I don't want us to duplicate the NUnit features and support each attribute explicitly. We should think about way when NUnit does all the work, and we simply ask it about that.

It could make sense to use some alternative approach. For instance, API could look like following:

[Test]
public void SomeTest([Values(1,2,3)]  int seed, [AutoValue] string generated)
{ }

However, I'm not sure it will work fine with discovery/execution, as value should be contant.

As a bottom line, I'm totally non-confident in the right approach (mostly because I'm not using NUnit). I don't want to create a mess in the code and make library usage over-complicated. Probably, we should make a good R&D to see all the options we have.

I like the [AutoValue] approach, I wonder what the implications would be if we had two parameters with the attribute: do we incur into the risk fo generating the same value twice for the same atomic type (say Int32?)

@Kralizek Probably. I'm just trying to say about alternative ways and that a huge investigation is required here to make this happen, rather than an ad-hoc fix.

Anyway, one of my working theories was not to duplicate how to handle the NUnit attributes, rather try to make sure NUnit sees the fixture'd arguments as "constant values".

e.g.

[Test, AutoData]
public void SomeTest([Values(1,2,3)] int seed, string generated) {}

is intercepted by AutoFixture and "transformed" into something equivalent of

[Test]
[TestCase("some-random-string-for-second-argument")]
public void SomeTest([Values(1,2,3)] int seed, string generated) {}

This would give us the following executions

• 1, "some-random-string-for-second-argument"
• 2, "some-random-string-for-second-argument"
• 3, "some-random-string-for-second-argument"

It could make a trick if you find the way to implemente that 😉 Previously I didn't see how that could be achieved, but I don't use NUnit, so I could easily overlook something 😕

This suggestion from @zvirja might be possible.

[Test]
public void SomeTest([Values(1,2,3)]  int seed, [AutoValue] string generated)
{ }

Using a custom NUnit attribute, here is the interface for a parameter attribute, the method requires an IEnumerable to be returned, https://github.com/nunit/docs/wiki/IParameterDataSource-Interface

Having a look at the current AutoData attribute, it is using the ITestBuilder interface which also returns an IEnumerable
Here is the current implementation https://github.com/AutoFixture/AutoFixture/blob/master/Src/AutoFixture.NUnit3/AutoDataAttribute.cs
https://github.com/nunit/docs/wiki/ITestBuilder-Interface

So this might be a viable solution to move the AutoData to a Parameter attribute.

Following @steph4nc line, I created this gist

The small unit test I created is correctly executed and interacts nicely with the other NUnit attributes (in the example I used Values and Range).

Users should be able to inherit from this attribute and create their own version with a customized Fixture.

Notable to report, the object is created only once and used for all the iterations.

Here are some screenshots from Visual Studio 2017 Enterprise

ReSharper test session

Live testing

Console

@zvirja sorry for the unsollecited mention but the default sorting in GitHub is "newest" issues and this slipped in page 2. :)

@Kralizek Thanks for investigation of the idea 👍 It looks like AutoData attributes in your gist are isolated to each other within the single test run. Will features like Frozen attribute work there?

@zvirja I'm not sure how Frozen works but I a parameter on each attribute should have its own independent instance and its own fixture with it.

To give it a try, I created an attribute deriving from the AutoValue one

public class FrozenAutoValueAttribute : AutoValueAttribute
{
    public FrozenAutoValueAttribute() : base(CreateFixture) { }

    private static IFixture CreateFixture()
    {
        IFixture fixture = new Fixture();

        fixture.Freeze<Model>();
        fixture.Freeze<string>();

        return fixture;
    }
}

And these two tests

[Test]
public void Two_AutoValueAttribute_should_share_same_frozen_complex_type(
    [FrozenAutoValue] Model first,
    [FrozenAutoValue] Model second
)
{
    Assert.That(first.Text, Is.EqualTo(second.Text));
    Assert.That(first.Value, Is.EqualTo(second.Value));
}

[Test]
public void Two_AutoValueAttribute_should_share_same_frozen_simple_type(
    [FrozenAutoValue] string first,
    [FrozenAutoValue] string second
)
{
    Assert.That(first, Is.EqualTo(second));
}

The first one is not passing, as expected.
The second one is more interesting. For some reason the AutoValueing simple types breaks NUnit and no test is returned even if the attribute itself behaves as expected. I will open an issue at NUnit today or tomorrow.

[Test] // this test is never executed nor detected by NUnit
public void AutoValueAttribute_should_provide_a_simple_type([AutoValue] int value)
{
    Assert.Pass();
}

Finally, a small test to validate the behavior of the attribute alone. This one passes without a glitch.

[Test]
[InlineAutoData(typeof(int))]
[InlineAutoData(typeof(string))]
[InlineAutoData(typeof(Model))]
public void AutoValueAttribute_generates_a_model(Type typeToGenerate, AutoValueAttribute sut)
{
    Mock<IParameterInfo> mockParameter = new Mock<IParameterInfo>();

    mockParameter.SetupGet(p => p.ParameterType).Returns(typeToGenerate);

    var values = sut.GetData(mockParameter.Object);

    Assert.That(values, Has.Exactly(1).InstanceOf(typeToGenerate));
}

I'm not sure how Frozen works but I a parameter on each attribute should have its own independent instance and its own fixture with it.

Sounds like an issue, as Frozen attribute is intended to work across different arguments. Here is a quick demo how it works:

[Theory, AutoData]
public void FrozenAttributeDemo([Frozen] Source source, ValueReader reader)
{
    source.Value = 42;

    var result = reader.GetValue();

    Assert.Equal(42, result);
}

public class Source
{
    public int Value { get; set; }
}

public class ValueReader
{
    private readonly Source _source;

    public ValueReader(Source source)
    {
        _source = source;
    }

    public int GetValue() => _source.Value;
}

It's important that ValueReader gets the same instance of source as provided via the parameter. Otherwise, this test will never pass.

When we are investigating new opportunities, we should ensure that Frozen attribute works, as it's a major building block during unit test writing.

I finally was able to find a solution to this issue in a way that satisfies the many requirements.

At the moment I have only a LinqPad script with the needed classes but I plan to make a PR out of it. The only issue is that it required few breaking changes and I'm discussing with @aivascu how to go forward.

I have focused on the AutoDataAttribute for now but I don't think it will take long to bring the same modifications to InlineAutoDataAttribute.

The new AutoDataAttribute generates as many tests as the combination of the inputs of the method parameters. If a parameter is given no value from a value generator, AutoFixture is used. Also, it's possible to freeze a value generated from the NUnit attributes.

Here are some examples. For each example, I show a LinqPad Dump of the generated test runs.

[Test]
[NUnitAutoData]
public void TestMethod1([Values(1,2)] int intValue, [Values] bool boolValue, string stringValue, [Range(10,11)] int anotherIntValue) { }

[Test]
[NUnitAutoData]
public void TestMethod2([Values(1,2,3), Frozen] int intValue, [Frozen] string stringValue, TestWithDependency dependency) { }

[Test]
[NUnitAutoData]
public void TestMethod3([ValueSource(nameof(SourceMethod))] int intValue, [Frozen] string stringValue, TestWithDependency dependency) { }

As you can see from the snippets, the Values, Range and ValueSource are used to generate individual test runs. Also, the values they provide are used to name the test so that each test run has a unique name.

What's left to do:

• Support InlineAutoData, required
• Refine support types naming, required
• Add support for combination strategies other than Combinatorial like Pairwise and Sequential, if needed
• Support scenarios of ValueSource with methods with parameters fed by AutoFixture, not sure if possible

EDIT:

• I forgot to attach the file. NUnitAutoDataAttribute.linq.txt (remove the .txt extension and open with LinqPad)
• I just realized that the Frozen support is flaky at best. I'm working on it.

@Kralizek I haven't gotten too deep into the review of your PR but it seems there is some overlapping work with #1257. I have also started a redesign of the AutoData attributes for NUnit 3, mostly in order to implement ClassAutoData and MemberAutoData. I'll analyze your solution to see how we can have both features implemented.

@aivascu I noticed your PR as well when I pushed mine. Am I wrong saying that your PR is to support a use case equivalent to the TestCaseSourceAttribute?

PS: What is CassAutoData? 🤔

@Kralizek indeed, I event named the attribute AutoTestCaseSourceAttribute, so it would be consistent with the NUnit naming. However this proved to not be a good decision in terms of respecting the SRP. Now I'm considering following the XUnit naming of attributes everywhere because that way the responsibilities are separated a bit better not to mention the attributes contain less code and are easier to understand and maintain.

PS: What is CassAutoData? 🤔

Sorry that's a typo. It's ClassAutoDataAttribute.

If you want to have a chat/call on discord, I'm there right now and later today after 14 CEST