Tag: Unit Testing

Migrating from MSTest to XUnit 2

We recently migrated most of our testing from the MSTest framework1 to XUnit 2 (from here on in, I will be referring to this as just XUnit). This was not a change taken lightly since it touched a lot of files, but we were motivated by a number of XUnit features, including reduced need to attribute test classes, easier data-driven tests, and parallel test execution.

Sadly, if you try this you may discover as we did that the XUnit documentation is equal parts super helpful and woefully lacking, depending on what you are trying to do. After hearing yet another colleague lament how hard it was to find information on some feature or other of XUnit, I thought it might be a good idea to document some of the things I have learned and hopefully, introduce yet another helpful XUnit resource to the Internet2.

For those not familiar with XUnit, the basics are pretty easy. In fact the existing XUnit documentation includes a handy table mapping concepts in other test frameworks to their XUnit equivalents. You can check that table out for details, but basically, through the use of attributes, constructors, IDisposable, and other interfaces, XUnit uses what I would describe as a more natural approach than other frameworks to concepts like tests, test initialization and cleanup, and test fixtures. Of course, this means that migrating from one framework to XUnit involves a bunch of file editing, but fear not for there is help.

XUnitConverter

The bulk of the migration was made a lot easier by using the XUnitConverter, a tool available in the dotnet/codeformatter GitHub repository. Although it does not take care of everything (beware if you have multiple test classes per file) and, depending on your preferred code format, can mess your formatting up a bit, but it does make the migration a lot easier.

The XUnitConverter runs against a csproj file. You can use PowerShell to recurse your solution and process all your projects like this:

Get-ChildItem -Recurse *.csproj | %{XUnitConverter $_.FullName}

Once the converter has done its thing, it is easy to identify further changes by using the compiler (things don't like to build if something did not work right). Although most things get converted with ease—[TestMethod] becomes [Fact], [TestInitialize]  becomes a constructor, complex tests will need a little more assistance to fully migrate. For example, XUnit uses interfaces and fixture classes to replace the kind of shared initialization and cleanup that MSTest provides via the [ClassInitialize] and [ClassCleanup]. We will start tackling these issues next time.

  1. Version 1 of MSTest, not the new and improved MSTest version 2 [↩]
  2. I would like to document my anecdotal information before I even consider tackling something a little more structured like contributing to the official documentation [↩]

Unit testing attribute driven late-binding

I've been working on a RESTful API using ASP WebAPI. It has been a great experience so far. Behind the API is a custom framework that involves some late-binding. I decorate certain types with an attribute that associates the decorated type with another type1. The class orchestrating the late-binding takes a collection of IDecorated instances. It uses reflection to look at their attributes to determine the type they are decorated with and then instantiates that type.

It's not terribly complicated. At least it wasn't until I tried to test it. As part of my development I have been using TDD, so I wanted unit tests for my late-binding code, but I soon hit a hurdle. In mocking IDecorated, how do I make sure the mocked concrete type has the appropriate attribute?

var mockedObject = new Mock();

// TODO: Add attribute

binder.DoSpecialThing( mockedObject.Object ).Should().BeAwesome();

I am using Moq for my mocking framework accompanied by FluentAssertions for my asserts2. Up until this point, Moq seemed to have everything covered, yet try as I might I couldn't resolve this problem of decorating the generated type. After some searching around I eventually found a helpful Stack Overflow question and answer that directed me to TypeDescriptor.AddAttribute, a .NET-framework method that provides one with the means to add attributes at run-time!

var mockedObject = new Mock();

TypeDescriptor.AddAttribute(
    mockedObject.Object.GetType(),
    new MyDecoratorAttribute( typeof(SuperCoolThing) );

binder.DoSpecialThing( new [] { mockedObject.Object } )
    .Should()
    .BeAwesome();

Yes! Runtime modification of type decoration. Brilliant.

So, why didn't it work?

My binding class that I was testing looked a little like this:

public IEnumerable<Blah> DoSpecialThing( IEnumerable<IDecorated> decoratedThings )
{
    return from thing in decoratedThings
           let converter = GetBlahConverter( d.GetType() )
           where d != null
           select converter.Convert( d );
}

private IConverter GetBlahConverter( Type type )
{
    var blahConverterAttribute = Attribute
        .GetCustomAttributes( type, true )
        .Cast<BlahConverterAttribute>()
        .FirstOrDefault();

    if ( blahConverterAttribute != null )
    {
        return blahConverterAttribute.ConverterType;
    }

    return null;
}

Looks fine, right? Yet when I ran it in the debugger and took a look, the result of GetCustomAttributes was an empty array. I was stumped.

After more time trying different things that didn't work than I'd care to admit, I returned to the StackOverflow question and started reading the comments; why was the answer accepted answer when it clearly didn't work? Lurking in the comments was the missing detail; if you use TypeDescriptor.AddAttributes to modify the attributes then you have to use TypeDescriptor.GetAttributes to retrieve them.

I promptly refactored my code with this detail in mind.

public IEnumerable<Blah> DoSpecialThing( IEnumerable<IDecorated> decoratedThings )
{
    return from thing in decoratedThings
           let converter = GetBlahConverter( d.GetType() )
           where d != null
           select converter.Convert( d );
}

private IConverter GetBlahConverter( Type type )
{
    var blahConverterAttribute = TypeDescriptor
        .GetAttributes( type )
        .OfType<BlahConverterAttribute>()
        .FirstOrDefault();

    if ( blahConverterAttribute != null )
    {
        return blahConverterAttribute.ConverterType;
    }

    return null;
}

Voila! My test passed and my code worked. This was one of those things that had me stumped for longer than it should have. I am sharing it in the hopes of making sure there are more hits when someone else goes Internet fishing for help. Now, I'm off to update Stack Overflow so that this is clearer there too.

  1. Something similar to TypeConverterAttribute usage in the BCL [↩]
  2. Though I totally made up the BeAwesome() assertion in this blog post [↩]