Tag: Versioning

Prerelease packages with branches, Appveyor, and MyGet

We use a workflow where Appveyor is setup for CI and then it automatically pushes newly built nuget packages to a MyGet feed for consumption. Perhaps in a future post, I will go over how to set all this up, but for now, let's assume you already have this working; you push changes to a branch in your GitHub repo, which then gets built and tested on Appveyor, before being pushed to MyGet. Everything is nice and smooth.

Unfortunately, the magic ended there. Since there is no differentiation between pushing prerelease changes and release changes, I found that I would either have to limit what branches built in on Appveyor or spend a lot of time curating MyGet to remove intermediate builds I did not want used. I knew that MyGet supported prerelease packages but no matter what I tried, I could not get Appveyor to build them. Unsurprisingly, I found this frustrating. Then I stumbled on this particular answer on StackOverflow:

However, there were some issues I had with this.

  1. It seemed wrong that I had to use an after_build or on_success step to explicitly build my nuget package
  2. I didn't want every build to be prerelease
  3. It didn't work

The first point smelled enough that I wanted to see if I could not have to do that, and that second point seemed really important.

So, I delved a little deeper and discovered that the nuspec file, which has a handy $version$ substitution for the version takes that information from the value of the AssemblyInformationalVersion attribute, which I did not have declared in my AssemblyInfo.cs. Since it was not in there, the Appveyor step declared to patch it did not do anything. This was easy to fix, so I edited my AssemblyInfo.cs to include the attribute and tried again. This time the version updated as I wanted, even without the after_build or on_success shenanigans.

However, it still was not quite right since now, every build being performed was marked as prerelease. While this is a potential workflow, where the appveyor.yml is updated when finally reaching release, what I wanted was for releases to occur when I tagged a branch. For that, I looked at tweaking how the Appveyor build version updated and what environment variables Appveyor defined that I could leverage.

It turns out that Appveyor defines APPVEYOR_REPO_TAG, which is set to true if the build was started by a tag being pushed. It also defines APPVEYOR_REPO_BRANCH containing the name of the branch being built. Armed with these two variables, I updated my appveyor.yml to have two init scripts.

init:
- ps: $env:customnugetversion = if ($env:APPVEYOR_REPO_TAG -eq $True) { "$env:APPVEYOR_BUILD_VERSION" } else { "$env:APPVEYOR_BUILD_VERSION-$env:APPVEYOR_REPO_BRANCH" }
- ps: Update-AppveyorBuild -Version $env:customnugetversion

The first script creates a new environment variable. If the APPVEYOR_REPO_TAG is set to true, the new variable gets set to the value of APPVEYOR_BUILD_VERSION; if not, it is set to APPVEYOR_BUILD_VERSION-APPVEYOR_REPO_BRANCH. So, for example, if the build was going to be version 2.4.0, it was not a tag, and the branch was master, then the new variable would be set to 2.4.0-master; however, if it was a tag, it would just be 2.4.0.

The second script calls the Update-AppveyorBuild cmdlet provided by Appveyor, passing the value of the new environment variable as the -Version parameter value.

These two init scripts, plus the AssemblyInformationalVersion attribute in the AssemblyInfo.cs (and corresponding assembly_information_version field under the assembly_info section of the appveyor.yml) were all I needed. Now, whenever I push to a branch, I get a new prerelease nuget package that I can use in my development coding, and whenever I create a new tag, I get a release package instead. Not only does this reduce my need to manually manage my nuget packages on MyGet, but it also means I can take advantage of the different retention policy settings between prerelease and release packages.

All in all, I find this workflow much nicer than what I had before. Hopefully some of you do too. Examples of the appveyor.yml file and associated AssemblyInfo.cs change can be seen in the following Gist.

Getting Information About Your Git Repository With C#

During a hackathon not so long ago, I wanted to incorporate some source control data into my .NET assembly version information for the purposes of troubleshooting installations, making it easier for people to report the code in which they found a bug, and making it easier for people to find the code in which a bug was found1. The plan was to automatically encode the branch, the commit hash, and whether there were local commits or local changes into the `AssemblyConfiguration` attribute of my assemblies during the build.

At the time, I hacked together the `RepositoryInformation` class below that wraps the command line tool to extract the required information. This class supported detecting if the directory is a repository, checking for local commits and changes, getting the branch name and the name of the upstream branch, and enumerating the log. Though it felt a little wrong just wrapping the command line (and seemed pretty fragile too), it worked. Unfortunately, it was dependent on git being installed on the build system; I would prefer the build to get everything it needs using package management like NuGet and npm2.

class RepositoryInformation : IDisposable
{
    public static RepositoryInformation GetRepositoryInformationForPath(string path, string gitPath = null)
    {
        var repositoryInformation = new RepositoryInformation(path, gitPath);
        if (repositoryInformation.IsGitRepository)
        {
            return repositoryInformation;
        }
        return null;
    }
    
    public string CommitHash
    {
        get
        {
            return RunCommand("rev-parse HEAD");
        }
    }
    
    public string BranchName
    {
        get
        {
            return RunCommand("rev-parse --abbrev-ref HEAD");
        }
    }
    
    public string TrackedBranchName
    {
        get
        {
            return RunCommand("rev-parse --abbrev-ref --symbolic-full-name @{u}");
        }
    }
    
    public bool HasUnpushedCommits
    {
        get
        {
            return !String.IsNullOrWhiteSpace(RunCommand("log @{u}..HEAD"));
        }
    }
    
    public bool HasUncommittedChanges
    {
        get
        {
            return !String.IsNullOrWhiteSpace(RunCommand("status --porcelain"));
        }
    }
    
    public IEnumerable<string> Log
    {
        get
        {
            int skip = 0;
            while (true)
            {
                string entry = RunCommand(String.Format("log --skip={0} -n1", skip++));
                if (String.IsNullOrWhiteSpace(entry))
                {
                    yield break;
                }
                
                yield return entry;
            }
        }
    }
    
    public void Dispose()
    {
        if (!_disposed)
        {
            _disposed = true;
            _gitProcess.Dispose();
        }
    }
    
    private RepositoryInformation(string path, string gitPath)
    {
        var processInfo = new ProcessStartInfo
        {
            UseShellExecute = false,
            RedirectStandardOutput = true,
            FileName = Directory.Exists(gitPath) ? gitPath : "git.exe",
            CreateNoWindow = true,
            WorkingDirectory = (path != null && Directory.Exists(path)) ? path : Environment.CurrentDirectory
        };
        
        _gitProcess = new Process();
        _gitProcess.StartInfo = processInfo;
    }
    
    private bool IsGitRepository
    {
        get
        {
            return !String.IsNullOrWhiteSpace(RunCommand("log -1"));
        }
    }
    
    private string RunCommand(string args)
    {
        _gitProcess.StartInfo.Arguments = args;
        _gitProcess.Start();
        string output = _gitProcess.StandardOutput.ReadToEnd().Trim();
        _gitProcess.WaitForExit();
        return output;
    }
    
    private bool _disposed;
    private readonly Process _gitProcess;
}

If I were to approach this again today, I would use the LibGit2Sharp NuGet package or something similar3. Below is an updated version of `RepositoryInformation` that uses LibGit2Sharp instead of git command line. Clearly, you could forego any type of wrapper for LibGit2Sharp and I probably would if I were incorporating this into a bigger task like the one I originally had planned.

class RepositoryInformation : IDisposable
{
    public static RepositoryInformation GetRepositoryInformationForPath(string path)
    {
        if (LibGit2Sharp.Repository.IsValid(path))
        {
            return new RepositoryInformation(path);
        }
        return null;
    }
    
    public string CommitHash
    {
        get
        {
            return _repo.Head.Tip.Sha;
        }
    }
    
    public string BranchName
    {
        get
        {
            return _repo.Head.Name;
        }
    }
    
    public string TrackedBranchName
    {
        get
        {
            return _repo.Head.IsTracking ? _repo.Head.TrackedBranch.Name : String.Empty;
        }
    }
    
    public bool HasUnpushedCommits
    {
        get
        {
            return _repo.Head.TrackingDetails.AheadBy > 0;
        }
    }
    
    public bool HasUncommittedChanges
    {
        get
        {
            return _repo.RetrieveStatus().Any(s => s.State != FileStatus.Ignored);
        }
    }
    
    public IEnumerable<Commit> Log
    {
        get
        {
            return _repo.Head.Commits;
        }
    }
    
    public void Dispose()
    {
        if (!_disposed)
        {
            _disposed = true;
            _repo.Dispose();
        }
    }
    
    private RepositoryInformation(string path)
    {
        _repo = new Repository(path);
    }

    private bool _disposed;
    private readonly Repository _repo;
}

I have yet to use any of this outside of my hackathon work or this blog entry, but now that I have resurrected it from my library of coding exploits past to write about, I might just resurrect the original plans I had too. Whether that happens or not, I hope you found this useful or at least a little interesting; if so, or if you have some suggestions related to this post, please let me know in the comments.

  1. Sometimes, like a squirrel, you want to know which branch you were on [↩]
  2. I had looked at NuGet packages when I was working on the original hackathon project, but had decided not to use one for some reason or another (perhaps the available packages did not do everything I wanted at that time) [↩]
  3. PowerShell could be a viable replacement for my initial approach, but it would suffer from the same issue of needing git on the build system; by using a NuGet package, the build includes everything it needs [↩]

Resolving XML references from embedded resources

Recently, I wanted to validate some XML via an XSD schema. Due to some product constraints and intentions regarding versioning, the schema is an embedded resource and is referenced via the noNamespaceSchemaLocation attribute.

When loading XML in .NET, you can specify an XmlResolver via the XmlReaderSettings . As stated in MSDN, the default uses a new XmlUrlResolver without credentials. This works great when the file is local on disk, but not when it is squirreled away inside my resources.

What I needed was a special version of XmlResolver that understood how to find my embedded schemas. So I created a derivation, XmlEmbeddedResourceResolver, to do just that.

public class XmlEmbeddedResourceResolver : XmlUrlResolver
{
	public XmlEmbeddedResourceResolver( Assembly resourceAssembly )
	{
		_resourceAssembly = resourceAssembly;
	}

	public override object GetEntity( Uri absoluteUri, string role, Type ofObjectToReturn )
	{
		if ( absoluteUri == null ) throw new ArgumentNullException( "absoluteUri", "Must provide a URI" );

		if ( ShouldAttemptResourceLoad( absoluteUri, ofObjectToReturn ) )
		{
			var resourceStream = GetSchemaStreamFromEmbeddedResources( absoluteUri.AbsolutePath );
			if ( resourceStream != null )
			{
				return resourceStream;
			}
		}
		return base.GetEntity( absoluteUri, role, ofObjectToReturn );
	}

	public override async Task<object> GetEntityAsync( Uri absoluteUri, string role, Type ofObjectToReturn )
	{
		if ( absoluteUri == null ) throw new ArgumentNullException( "absoluteUri", "Must provide a URI" );

		if ( ShouldAttemptResourceLoad( absoluteUri, ofObjectToReturn ) )
		{
			var resourceStream = await GetSchemaStreamFromEmbeddedResourcesAsync( absoluteUri.AbsolutePath );
			if ( resourceStream != null )
			{
				return resourceStream;
			}
		}

		return await base.GetEntityAsync( absoluteUri, role, ofObjectToReturn );
	}

	private static bool ShouldAttemptResourceLoad( Uri absoluteUri, Type ofObjectToReturn )
	{
		return ( absoluteUri.Scheme == Uri.UriSchemeFile && ofObjectToReturn == null || ofObjectToReturn == typeof( Stream ) || ofObjectToReturn == typeof( object ) );
	}

	private Stream GetSchemaStreamFromEmbeddedResources( string uriPath )
	{
		var schemaFileName = Path.GetFileName( uriPath );
		var schemaResourceName = _resourceAssembly.GetManifestResourceNames().FirstOrDefault( n => n.EndsWith( schemaFileName ) );
		if ( schemaResourceName != null )
		{
			return _resourceAssembly.GetManifestResourceStream( schemaResourceName );
		}
		return null;
	}

	private Task<object> GetSchemaStreamFromEmbeddedResourcesAsync( string uriPath )
	{
		return Task.Run( () => (object)GetSchemaStreamFromEmbeddedResources( uriPath ) );
	}

	private readonly Assembly _resourceAssembly;
}

When asked to find a file-based reference, this steps in and looks in embedded resources first for a file of the same name. Since the file could be namespaced anywhere in the resources, I opted to look for any resource in any namespace with the same file name. If it is there, it loads it, otherwise it defers to the base implementation. This means there is no easy way to override the embedded file with a local one; however, that could be redressed by calling the base implementation first and then only searching embedded resources if that failed.

Note that I also implemented the async methods. I am certain my implementation is a little naive, but it generally works. Just be careful if you allow this to be used asynchronously as I discovered you can very easily create a deadlock when used in conjunction with locks. This is not necessarily a caveat of my implementation, but of asynchronous programming in general.

Hopefully, others will find this useful. Let me know in the comments if you use this or something similar.

Versioning with T4 templates

In the last month or two I've started using T4 templates, a feature originally made available in Visual Studio 2005 though rarely used outside of code generation tools for database models and the like. Though I have known about T4 templates for quite some time, like many developers, I have struggled to conceive of a practical application in my every day software development activities. That is until recently, when two uses came along at once.

1 day, 7000 lines

The first use came when I started to implement a strongly typed unit conversion framework. I needed to generate types for different quantities (Area, Distance, Volume, etc.), units and operations that convert between them. As I was coding my framework, I noticed that much of the code was identical. Rather than cut and paste a lot (I had over 20 different quantities and hundreds of units) and deal with a horribly tedious task when bug fixing, I realised that a T4 template and an XML file would reduce my effort a lot. I embarked on learning all about T4 and within a day I was auto-generating over 7000 lines of useful code from far less than that.

I am still working on some nitty gritty details in that framework so I'd rather not blog about it in detail just yet. However, having become more familiar with T4 templates, it was much easier to spot other places they could add value. The next opportunity arose when working on another project. One of the first things I tend to do when setting up a development project is set up the versioning. I split common attributes shared by every assembly in my solution into a different file and then link to it in each project.

[assembly: System.Reflection.AssemblyCompany("SomewhatAbstract.com")]
[assembly: System.Reflection.AssemblyProduct("DoobreyFlap")]
[assembly: System.Reflection.AssemblyCopyright("Copyright © 2012 SomewhatAbstract.com")]

[assembly: System.Reflection.AssemblyConfiguration("alpha")]

// Version information for an assembly consists of the following four values:
//
// Major Version
// Minor Version
// Patch
// Revision
//
[assembly: System.Reflection.AssemblyVersion("1.0.0.101")]
[assembly: System.Reflection.AssemblyFileVersion("1.0.0.101")]

This single file is then updated by a script (MSBuild step or pre-build BAT or PS file) based on information from my source control provider of choice. You may wonder where T4 fits in here. Well, it doesn't, yet.

The next step in a development project is to consider deployment. I use Windows Installer XML (aka WIX) for my installers, an open source installation framework that uses XML files to describe the installer. Just as with my source code, I split the version information into its own file as this simplifies update of that information and allows me to share it across different installations. I also update some of this information using a script.

<?xml version="1.0" encoding="utf-8"?>

<!-- These values should be updated when a new release is made. This ensures that the new version can
 upgrade the old. -->

<!-- This is the product version.
 Note: only the first 3 tokens are used (i.e. 1.0.0.0 will be ignored). -->
<?define ProductVersion=1.0.0?>

<?define ProductVersionText="1.0.0.101"?>
<?define ProductState="alpha"?>

<Include />

So, now I have two version files. My scripts update these to include revision information from my source control provider (if you are familiar with semantic versioning, this goes in the build version, though I admit, in my examples I am not using SemVer). However, when doing a new major, minor or patch update, I have to manually edit each of these files to change the major, minor or patch version. On more than one occasion, I've updated one file and not the other. This only gets more complicated if there are additional files that need versioning (maybe there's also VB projects or some markdown for a release notice). This is where T4 templates comes in handy.

1 file, many files

Now, some of you may think that this is where my plan falls down. Can't T4 templates only output one file? How can one file be suitable for C#, VB and XML all at the same time? The answer is, it can't. At least not without a little help.

As you can execute any code in a T4 template, you could just write out a file using a FileStream or something similar. However, while researching T4 templates for my unit conversion framework and realising that 7000 lines of code and tens of different types in a single file might be a tad unmanageable in some circumstances, I discovered a rather handy T4 mix-in that permits me to output multiple files from a single template using a simple syntax.

Using Damien Guard's Manager.ttinclude file, I created a T4 template and outputted my C# and WIX files. Now, updating the version just means updating the template. This could easily be extended so that the information is read from some other file (such as an XML or JSON file) rather than hard-coded in the template. In addition, the revision information that I extract from source control via script could be extracted via the template itself, if I so choose.

<#@ template debug="true" hostSpecific="true"
#><#@ output extension=".cs"
#><#@ include file="Manager.ttinclude"
#><# // Create the file manager so we can output multiple files.
#><# var manager = Manager.Create(this.Host, this.GenerationEnvironment);
#><#
#><# // Setup the version information.
#><# var version = new Version(1, 0, 0, 101);
#><# var configuration = "alpha";
#><# var company = "SomwhatAbstract.com";
#><# var product = "DoobreyFlap";
#><# var copyright = "Copyright © 2012 SomewhatAbstract.com";
#><#
#><# // Output the C# versioning
#><# //
#><# // VersionInfo.g.cs
#><# //
#>
[assembly: System.Reflection.AssemblyCompany(<#= company #>)]
[assembly: System.Reflection.AssemblyProduct(<#= product #>)]
[assembly: System.Reflection.AssemblyCopyright(<#= copyright #>)]

[assembly: System.Reflection.AssemblyConfiguration(<#= configuration #>)]

// Version information for an assembly consists of the following four values:
//
// Major Version
// Minor Version
// Patch
// Revision
//
[assembly: System.Reflection.AssemblyVersion(<#= version.ToString() #>)]
[assembly: System.Reflection.AssemblyFileVersion(<#= version.ToString() #>)]
<#
#><#
#><#
#><# // Output the WIX versioning
#><# //
#><# // VersionInfo.g.wxi
#><# //
#><# manager.StartNewFile("VersionInfo.g.wxi");
#>
<?xml version="1.0" encoding="utf-8"?>

<!-- These values should be updated when a new release is made. This ensures that the new version can
 upgrade the old. -->

<!-- This is the product version.
 Note: only the first 3 tokens are used (i.e. 1.0.0.0 will be ignored). -->
<?define ProductVersion=<#= version.ToString(3) #>?>

<?define ProductVersionText="<#= version.ToString() #>"?>
<?define ProductState="<#= configuration #>"?>

<Include />
<# manager.EndBlock();
#><# manager.Process(true);
#><#+
#>

This template will output two files: VersionInfo.g.cs and VersionInfo.g.wxi. Note that the C# file is named after the template whereas the WIX file is named on line 41. I recommend adding the g suffix to make it easier to see that these are auto-generated files. I have this template in a special PreSolutionBuild project that sits at the start of the dependency chain for my solution. This keeps things nice and tidy in my solution.

Conclusion

As you can see, this is a very simple use for T4 templates, yet it solves a reasonably common issue. In the past I may have addressed this issue of making sure the version files are up-to-date by adding to the release processes or creating a bespoke tool. Personally, I prefer the elegance of the T4-based approach and I hope that others find it just as useful as I do.

For more information on T4 templates, I found the following resources exceedingly useful: