Signing GitHub Commits With A Passphrase-protected Key and GPG2

GitHub recently added support for signed commits. The instructions for setting it up can be found on their website and I do not intend to rehash them here. I followed those instructions and they work splendidly. However, when I set mine up, I had used the version of GPG that came with my Git installation. A side effect I noticed was that if I were rebasing some code and wanted to make sure the rebased commits were still signed (by running git rebase with the -S option), I would have to enter my passphrase for the GPG key for every commit (which gets a little tedious after the first five or so).

Shows some commits on GitHub with the Verified indicator showing those that have been signed
How GitHub shows signed commits

Now, there are a couple of ways to fix this. One is easy; just don't use a passphrase protected key. Of course, that would make it a lot easier for someone to sign commits as me if they got my key file, so I decided that probably was not the best option. Instead, I did a little searching and found that GPG2 supports passphrase protected keys a little better than the version of GPG I had installed as part of my original git installation.

Using the GPG4Win website, I installed the Vanilla version1. I then had to export the key I had already setup with GitHub from my old GPG and import it into the new. Using gpg --list-keys, I obtained the 8 character ID for my key (the bit that reads BAADF00D in this example output):

Which I then used to export my keys from a Git prompt:

This gave me two files ( privatekey.txt and publickey.txt) containing text representations of the private and public keys.

Using a shell in the GPG2 pub folder ( "C:\Program Files (x86)\GNU\GnuPG\pub"), I then verified them (always a good practice, especially if you got the key from someone else) before importing them2:

And rather than give me details of the key, it showed me this error:

What was going on? I tried verifying it with the old GPG and it gave me a different but similar error:

I tried the public key export and it too gave these errors. It did not make a whole heap of sense. Trying to get to the bottom of it, I opened the key files in Visual Studio Code. Everything looked fine until I saw this at the bottom of the screen.

Encoding information from Visual Studio Code showing UTF16
Encoding information from Visual Studio Code

It turns out that Powershell writes redirected output as UTF-16 and I had not bothered to check. Thinking this might be the problem, I resaved each file as UTF-8 and tried verifying privatekey.txt again:

Success! Repeating this for the publickey.txt file gave the exact same information. With the keys verified, I was ready to import them into GPG2:

With the keys imported, I ran gpg --list-keys to verify they were there and then made sure to delete the text files.

Finally, to make sure that Git used the new GPG2 instead of the version of GPG that it came with, I edited my Git configuration:

Now, when I sign commits and rebases, instead of needing to enter my passphrase for each commit, I am prompted for the passphrase once. Lovely.


  1. You could also look at installing the command line tools from https://www.gnupg.org/download/ though I do not know if the results will be the same 

  2. Note that I am not showing the path to the file here for the sake of brevity, though I am sure you get the idea that you'll need to provide it 

Octokit and Noise Reduction with Pull Requests

Last time in this series on Octokit we looked at how to get the commits that have been made between one release and another. Usually, these commits will contain noise such as lazy commit messages and merge flog ("Fixed it", "Corrected spelling", etc.), merge commits, or commits that formed part of a larger feature change submitted via pull request. Rather than include all this noise in our release note generation, I want to filter those commits and either remove them entirely, or replace them with their associated pull request (which hopefully will be a little less noisy).

Before we filter out the noise, it seems prudent to reduce the commits to be filtered by matching them to pull requests. As with commits, we can query pull requests using a specific set of criteria; however, though we can request the results be sorted a certain way, we cannot specify a date range. To get all the pull requests that were merged before our release, we need to query for all the pull requests and then filter by date locally.

This query can be slow, since we are getting all closed pull requests in the repository. We could speed it up by providing a base branch name in the query criteria. However, to remove as much commit noise as possible, I would like to include pull requests that were merged to a different branch besides just the release branch1. We could make things more performant by managing a list of active release branches and then querying pull requests for each of those branches only rather than the entire repository, but for now, we will stick with the less optimal approach as it keeps the code examples a little cleaner.

Before we can start filtering our commits against the pull requests, we need to get the commits that comprise each pull request. When requesting a collection of items (like we did for pull requests), the GitHub API returns just enough information about each item so that we can filter and identify the ones we really care about. Before we can do things with other properties on the items, we have to request additional information. More information on each pull request can be obtained about a specific pull request by using the Get, Commits, Files, and Merged calls. The Get call returns the same type of objects as the GetAllForRepository method, except that all the data is now populated instead of just a few select properties; the Merged call returns a Boolean value indicating if the PR has been merged (equivalent to the Merged property populated by Get); the Files method returns the files changed by that pull request; and the Commits method returns the commits.

At this point, things are looking pretty good: we can get a list of commits in the release and a list of pull requests that might be in the release. Now, we want to filter that list of commits to remove items that are covered by a pull request. This is easy; we just compare the hashes and remove the matches.

Using the collection of commits for the latest release, we join the commits from the pull requests using the SHA hash and then select all release commits that have no matching commit in the pull requests2. However, we don't want to lose information just because we're losing noise, so we have to maintain a list of the pull requests that were matched so that we can build our release note history. To keep track, we will hold off on discarding any information by pairing up commits in the release with their prospective pull requests instead of just dropping them.

Going back to where we had a list of pull requests merged prior to our release, let us revisit getting the commits for those pull requests and this time, pairing them with the commits in the release to retain information.

Now we have a list of commits paired with their parent pull request, if there is one. Using this we can build a more meaningful set of changes for a release. If I run this on the latest release of the Octokit.NET repository and then group the commits by their paired pull request, I can see that the original list of 135 commits would be reduced to just 58 if each commit that belonged to a pull request were bundled into just one entry.

Next, we need to process the commits to remove those representing merges and other noise. These are things to discuss in the next post of this series where perhaps we will take stock and see whether this effort has been valuable in producing more meaningful release note generation. Until then, thanks for reading and don't forget to leave a comment.


  1. often changes are merged forward from one branch to another, especially if there are multiple release branches to support patch development and such 

  2. The join in this example is an outer join; we are taking the join results and using DefaultIfEmpty() to supply an empty collection when there was nothing to join 

Octokit and the Documentation Nightmare

Before I get into the meat of this series of posts, I would like to set the scene. Like many organisations that perform some level of software development these days, we use GitHub. Here at CareEvolution, some developers use the web interface extensively, some use the command line, and others use the GitHub desktop client1, but most use a combination of two or more, depending on the task. This works great for developers, who have each found a comfortable workflow for getting things done, but it is not so great for those involved with DevOps, QA, or documentation where there is a need to find out user-friendly details of what the developers did. Quite often, a feature or bug fix involves several commits and while each has a comment or two, and perhaps an associated pull request (PR) or issue has a general description, but there is no definitive list of "this is what release X contains" that can be presented to a customer. Not only that but sometimes a PR or issue is resolved in an earlier release and merged forward. While we have lists of what a release is going to include, quite often there is more detail that we would like to include, and we often have additional changes as we adapt to the changing requirements of our customers. All this means that one or more people end up trawling the commits, trying to determine what the changes are. It is not a happy task.

"There is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things."

Niccolo Machiavelli
The Prince (1532)

Now, I know that this could all be avoided if people documented changes more clearly, perhaps added release notes to commits, raised issues for documentation changes, or created release notes on the release when it is made. However, no matter how noble change may be, anyone who has worked in process definition for any length of time will know that changing the behaviour of people is the hardest task of all, and therefore it should be avoided unless absolutely necessary. It was with that in mind that I decided mining the existing data for information would be an easier first step than jumping straight to asking people to change. So, with the aim of making life a little easier, I started looking at ways to automate the trawling.

I figured that by throwing out noisy and typical developer non-descriptive commits like "fixed spelling" or "updated comment", and by combining commits under the corresponding PR or issue, I could create useful summary of changes. This would not be customer-ready, but it would be ready for someone to turn into a release note without needing to trawl git history. In fact, if I included details of who committed the changes, it might even provide a feedback loop that would improve the quality of developer commit messages; developers do not like interruptions, so anyone asking for more detail on a commit they made should start to reinforce that if they wrote better commits, PRs, issues, they would get less interruptions.

Octokitty2

Octokit .NET logoAfter a dismissing using git locally to perform this task (I figured those who might need this tool would probably not want to get the repository locally) and reading up on the GitHub API a little, I cracked open LINQPad —my tool of choice for hacking— and went looking for a Nuget package to help. It was during that search that I happily stumbled on Octokit, the official GitHub library for interacting with the GitHub API. At the time of writing, Octokit reflects the polyglot nature of GitHub users, providing variants for Ruby, .NET, and Objective C, as well as experimental versions for Python, and Go. I installed the Octokit Nuget package into LINQPad and started hacking (there is also a reactive version for IObservable fans).

Poking around the various objects, and reading some documentation on GitHub (Octokit is open source), I got a feel for how the library wrapped the APIs. Though, I had not yet got any code running, I was making progress. Confident that this would enable me to create the tool I wanted to create, I started writing some code to gather a list of releases for a specific repository and stumbled over my first hurdle; authentication. It turns out it is not quite as straight-forward as I thought (the days of username and password are quite rightly behind us3), and so, my adventure began.

And then…

This is a good place to stop for this week, I think. As the series progresses, I will be piecing together the various parts of my "release note guidance" tool and hopefully, end up with a .NET library to augment Octokit with some useful history mining functionality. Next time, we will take a look at authentication with Octokit (and there will be code).


  1. OSX and Windows variants 

  2. or, James Bond for kids 

  3. OK, that's a lie, but I want to encourage good behaviour 

Getting Information About Your Git Repository With C#

During a hackathon not so long ago, I wanted to incorporate some source control data into my .NET assembly version information for the purposes of troubleshooting installations, making it easier for people to report the code in which they found a bug, and making it easier for people to find the code in which a bug was found1. The plan was to automatically encode the branch, the commit hash, and whether there were local commits or local changes into the AssemblyConfiguration attribute of my assemblies during the build.

At the time, I hacked together the RepositoryInformation class below that wraps the command line tool to extract the required information. This class supported detecting if the directory is a repository, checking for local commits and changes, getting the branch name and the name of the upstream branch, and enumerating the log. Though it felt a little wrong just wrapping the command line (and seemed pretty fragile too), it worked. Unfortunately, it was dependent on git being installed on the build system; I would prefer the build to get everything it needs using package management like NuGet and npm2.

If I were to approach this again today, I would use the LibGit2Sharp NuGet package or something similar3. Below is an updated version of RepositoryInformation that uses LibGit2Sharp instead of git command line. Clearly, you could forego any type of wrapper for LibGit2Sharp and I probably would if I were incorporating this into a bigger task like the one I originally had planned.

I have yet to use any of this outside of my hackathon work or this blog entry, but now that I have resurrected it from my library of coding exploits past to write about, I might just resurrect the original plans I had too. Whether that happens or not, I hope you found this useful or at least a little interesting; if so, or if you have some suggestions related to this post, please let me know in the comments.


  1. Sometimes, like a squirrel, you want to know which branch you were on 

  2. I had looked at NuGet packages when I was working on the original hackathon project, but had decided not to use one for some reason or another (perhaps the available packages did not do everything I wanted at that time)  

  3. PowerShell could be a viable replacement for my initial approach, but it would suffer from the same issue of needing git on the build system; by using a NuGet package, the build includes everything it needs 

Getting posh-git in all your PowerShell consoles using GitHub for Windows

If you use git for version control and you use Microsoft Windows, you may well have used posh-git, a module for PowerShell. For those that have not, posh-git adds some git superpowers to your PowerShell console including tab completion for git commands, files and repositories, as well as an enhanced command prompt that tells you the current branch and its state1.

PowerShell console using posh-git
PowerShell console using posh-git
GitHub for Windows
GitHub for Windows

GitHub for Windows includes posh-git for its PowerShell console, if you choose that console when installing or later in the settings. It even adds a nice console icon to the task bar and Start screen2. Unfortunately, posh-git is only installed for the special version of the console that GitHub for Windows provides and you cannot make that prompt run as administrator, which can be useful once in a while.

Now, you could install a separate version of posh-git for all your other PowerShell needs, but that seems wrong. Especially since GitHub for Windows will happily keep its version up-to-date but you'd have to keep track of your other installation yourself.

Faced with this problem, I decided to hunt down how GitHub for Windows installed posh-git to see if I could get it into the rest of my PowerShell consoles. I quickly discovered ~\AppData\Local\GitHub containing both the posh-git folder and shell.ps1, the script that sets up the GitHub shell. The fantastic part of this script is that it sets up an environment variable for posh-git, github_posh_git, so you don't even need to worry about whether the folder changes3.

Armed with this information, you can edit your PowerShell profile4 and edit it to call both the GitHub shell script and the example profile script for posh-git5.

Once the edits are saved, close and reopen the PowerShell console to run the updated profile. Posh-git should now be available and all you have to do to keep it up-to-date is run the GitHub for Windows client once in a while.


  1. such as if there are any unstaged or uncommitted files and whether the branch is behind, ahead, or diverged from the remote 

  2. or menu, if you're pre-Windows 8 or installed something to bring the Start menu back 

  3. and if you've seen the folder name for posh-git in the GitHub for Windows installation, you'll see why that's useful 

  4. just enter notepad $profile at the PowerShell prompt 

  5. you may want to do the same thing for the PowerShell ISE, as it uses a separate profile script