TestMethod, TestInitialize, and TestCleanup in XUnit2

In the last post, I briefly described how to automatically migrate your MSTest tests to XUnit by using the XUnitConverterĀ utility. Of course, nothing is ever that simple; MSTest has some concepts that XUnit expresses very differently1 like how toĀ share code between tests whether that is setup, fixtures, cleanup, or data. Some of these concepts are implemented differently enough that automating the migration from one to the other would be very difficult if not impossible. However, some of it really is that simple. Before we look at the difficult examples, I thought it would be useful to illustrate how some of the simple concepts map from MSTest to XUnit using an example2.

So, let's look at an MSTest example (contrived, of course):

[TestClass]
public class MyTests
{
    private MemoryStream _stream;

    [TestInitialize]
    public void Initialize()
    {
        _stream = new MemoryStream();
    }

    [TestCleanup]
    public void Cleanup()
    {
        _stream.Dispose();
    }

    [TestMethod]
    public void Test1()
    {
        //Arrange
        //Act
        //Assert
    }

    [TestMethod]
    public void Test2()
    {
        //Arrange
        //Act
        //Assert
    }
}

Clearly, I cheated by not actually making the tests do anything, but the content of the test methods is mostly irrelevant; you set some stuff up, you do something, and you assert a resultā€“it's all the same regardless of the test framework. However, this is a simple example of a test class written for the MSTest framework. There are attributes to tell the framework that the class is a test class, which methods inside of it are test methods, and which methods should be called before and after each test. In this case, our test initialization creates a stream, which is then disposed of in the cleanup method; each test method would get sandwiched in the middle.

After converting to XUnit with the converter tool, the same class will look something like this:

public class MyTests : IDisposable
{
    private MemoryStream _stream;

    public MyTests()
    {
        _stream = new MemoryStream();
    }

    public void Dispose()
    {
        _stream.Dispose();
    }

    [Fact]
    public void Test1()
    {
        //Arrange
        //Act
        //Assert
    }

    [Fact]
    public void Test2()
    {
        //Arrange
        //Act
        //Assert
    }
}

There are a few things that happened.

  1. The class no longer has an attribute. XUnit knows the class is a test class because it contains tests3.
  2. The tests are decorated with a [Fact]Ā attribute, which is equivalent to [TestMethod].
  3. The [TestInitialize]Ā and [TestCleanup]Ā attributes are gone. Instead, the class constructor is used for test initialization and the DisposeĀ method along with deriving from IDisposableĀ indicates that there is test cleanup code.

Overall, I love how the XUnit syntax works with C# syntax and .NET idioms in declaring tests. Not only does this reduce the ceremony around defining tests by reducing the various decorators, but it also allows for cleaner coding practices. For example, we can now correctly mark our memory stream member variable as readonly.

private MemoryStream _stream;

By relying on C# syntax and standard interfaces in this way, the lifecycle of a test is clearer too; XUnit will construct and dispose the class for each test in the class, making it easy to see how each test will run. This idiomatic way of declaring tests allows for separation of concerns, keeping test classes light and focused. This will be illustrated when we later look at otherĀ concepts in MSTest like [ClassInitialize]Ā and [ClassCleanup], TestContext, and [DeploymentItem],Ā and how XUnit tackles the problems these concepts solved.

  1. and for good reasons, IMHO []
  2. XUnit documentation has a handy table but I don't think it's as illustrative as it could be []
  3. why MSTest did not make assumptions like this, I do not know []

Migrating from MSTest to XUnit 2

We recently migrated most of our testing from the MSTest framework1 to XUnit 2 (from here on in, I will be referring to this as just XUnit). This was not a change taken lightly since it touched a lot of files, but we were motivated by a number of XUnit features, including reduced need to attribute test classes, easier data-driven tests, and parallel test execution.

Sadly, if you try this you may discover as we did that the XUnit documentation is equal parts super helpful and woefully lacking, depending on what you are trying to do. After hearing yet another colleague lament how hard it was to find information on some feature or other of XUnit, I thought it might be a good idea to document some of the things I have learned and hopefully, introduce yet another helpful XUnit resource to the Internet2.

For those not familiar with XUnit, the basics are pretty easy. In fact the existing XUnit documentation includes a handy table mapping concepts in other test frameworks to their XUnit equivalents. You can check that table out for details, but basically, through the use of attributes, constructors, IDisposable, and other interfaces, XUnit uses what I would describe as a more natural approach than other frameworks toĀ concepts like tests, test initialization and cleanup, and test fixtures. Of course, this means that migrating from one framework to XUnit involves a bunch of file editing, but fear not for there is help.

XUnitConverter

The bulk of the migration was made a lot easier by using the XUnitConverter, a tool available in the dotnet/codeformatter GitHub repository. Although it does not take care of everything (beware if you have multiple test classes per file) and, depending on your preferred code format, can mess your formatting up a bit, but it does make the migration a lot easier.

The XUnitConverter runs against a csprojĀ file. You can use PowerShell to recurse your solution and process all your projects like this:

Get-ChildItem -Recurse *.csproj | %{XUnitConverter $_.FullName}

Once the converter has done its thing, it is easy to identify further changes by using the compiler (things don't like to build if something did not work right). Although most things get converted with easeā€”[TestMethod]Ā becomes [Fact], [TestInitialize]Ā  becomes a constructor, complex tests will need a little more assistanceĀ to fully migrate. For example, XUnit uses interfaces and fixture classes to replace the kind of shared initialization and cleanup that MSTest provides via the [ClassInitialize]Ā and [ClassCleanup]. We will start tackling these issues next time.

  1. Version 1 of MSTest, not the new and improved MSTest version 2 []
  2. I would like to document my anecdotal information before I even consider tackling something a little more structured like contributing to the official documentation []