Octokit, Merge Commits, and the Story So Far

In the last post we had reduced our commits by matching them against pull requests; next, we can look for noise in the commit message content itself. Although I have been using the Octokit.NET repository as the target for testing with its low noise, high quality commit messages, we can envisage a less consistent repository that has some noisy commits. For example, how often have you seen or written commit messages like "Fixed spelling", "Fixed bug", or "Stuff"1?

How we detect these noisy commits is important; if our filtering is too simple, we remove too many things and if it is too strict, we remove too few. Rather than go deep into one specific implementation, I just want to introduce the idea of filtering based on message content. In the long term, I think it would be interesting to apply learning algorithms,  but I'm sure some simple, configurable pattern matching should suffice2.

If I run the filtering I have described so far3 on the Octokit.NET latest release, this is what we get:

The value of this is clearer if we see the commit list before processing:

The work so far has reduced a list of 135 commits down to 58, and so far, it looks like we have not lost any really useful "release note"-worthy information. However, the eagle-eyed among you may noticed that our 58 messages contain duplicate information. This is because each pull request is listed twice; once for the pull request title I inserted in place of its individual commits, and again for the merge commit that merged that pull request. These merge commits are not filtered out because they do not belong to the commits inside the pull request. Instead, they are an artifact of merging the pull request4.

At first, I thought the handy MergeCommitSha property of the pull request would help, but it turns out this refers to a test merge and is to be deprecated5. Instead, I realised that the messages I wanted to remove all had "Merge pull request #" in them, followed by the pull request number. This seems like a perfect use case for our pattern matching filtering. Since we have the pull requests, we could use their numbers to match each merge message exactly, but I decided to do the simpler thing of excluding any message starting with "Merge pull request #".

Filtering for messages that begin with "Merge pull request #" gives us a shortlist of just 31 messages:

I think this is a pretty good improvement over the raw commit list. Combining this list with links back to the relevant commits and pull requests should enable someone to discern the content of a release note much faster than using the raw commit list alone. I will leave that as an exercise or perhaps a future post. As always, thanks for reading. If you find yourself using Octokit to trawl your own repositories for release note information, I would love to hear about it in the comments.


  1. We're all friends here, you can admit it 

  2. The filtering should be configurable so that we can tailor it to the repository we are processing 

  3. excluding the last step of filtering by message content 

  4. Perhaps stating the obvious 

  5. https://developer.github.com/v3/pulls/ 

XBOX One Screenshots as Windows 10 Desktop Backgrounds

From the mountain vistas of Far Cry 4 and Rise of the Tomb Raider, to the cityscapes of Grand Theft Auto 5 and Batman: Arkham Knight, many of the current generation console games are gorgeous. The XBOX One lets you capture these stunning scenes as a screenshot or video clip, which you can then share with your mates or completely forget about until something randomly reminds you they exist and you lose hours browsing them all, reminiscing about hilarious bugs or wondering why in the hell you decided to record what you just watched.

When I finally installed Windows 10 on my laptop and saw that OneDrive was integrated into the system, I had a flash of inspiration. I use the same Microsoft account on both my laptop and my XBOX which means they share the same OneDrive (among other things)1.

I went to Upload on my XBOX and shared one or two screenshots to OneDrive (one of the possible ways to share screenshots and clips). I then checked my laptop and after a few minutes, saw the newly shared images under the Pictures/Xbox Screenshots folder of OneDrive. It was only a few steps to get from that to having my screenshots as a desktop background slideshow.

Setting up background slideshow
Setting up background slideshow

To setup the slideshow, I hit Windows+I to get to Settings, then selected Personalization. Choosing the Background section on the left, I selected Slideshow from the Background dropdown on the right, then I browsed to the screenshots folder under OneDrive and selected it as the source of pictures for the slideshow.

Setting up automatic accent color
Setting up automatic accent color

To make sure things looked right with my backgrounds, I also chose the Colors personalization section and checked for Windows to automatically pick an accent color.

ExampleScreenshots

Now, whenever I share a screenshot to OneDrive from my XBOX, it automatically gets added to the desktop background slideshow on my laptop and desktop computers. Of course, I still have to remember to share it in the first place, but I am hoping some future system update will make sharing automatic or at least easier from within a game (like a Take Screenshot and Share feature in one go).

Thanks for stopping by and don't forget to leave a comment if you find this useful or have a story to share about how you make use of the XBOX sharing features.


  1. Using the same Microsoft account across devices and services really helps make the most of Microsoft's offerings 

Octokit and Noise Reduction with Pull Requests

Last time in this series on Octokit we looked at how to get the commits that have been made between one release and another. Usually, these commits will contain noise such as lazy commit messages and merge flog ("Fixed it", "Corrected spelling", etc.), merge commits, or commits that formed part of a larger feature change submitted via pull request. Rather than include all this noise in our release note generation, I want to filter those commits and either remove them entirely, or replace them with their associated pull request (which hopefully will be a little less noisy).

Before we filter out the noise, it seems prudent to reduce the commits to be filtered by matching them to pull requests. As with commits, we can query pull requests using a specific set of criteria; however, though we can request the results be sorted a certain way, we cannot specify a date range. To get all the pull requests that were merged before our release, we need to query for all the pull requests and then filter by date locally.

This query can be slow, since we are getting all closed pull requests in the repository. We could speed it up by providing a base branch name in the query criteria. However, to remove as much commit noise as possible, I would like to include pull requests that were merged to a different branch besides just the release branch1. We could make things more performant by managing a list of active release branches and then querying pull requests for each of those branches only rather than the entire repository, but for now, we will stick with the less optimal approach as it keeps the code examples a little cleaner.

Before we can start filtering our commits against the pull requests, we need to get the commits that comprise each pull request. When requesting a collection of items (like we did for pull requests), the GitHub API returns just enough information about each item so that we can filter and identify the ones we really care about. Before we can do things with other properties on the items, we have to request additional information. More information on each pull request can be obtained about a specific pull request by using the Get, Commits, Files, and Merged calls. The Get call returns the same type of objects as the GetAllForRepository method, except that all the data is now populated instead of just a few select properties; the Merged call returns a Boolean value indicating if the PR has been merged (equivalent to the Merged property populated by Get); the Files method returns the files changed by that pull request; and the Commits method returns the commits.

At this point, things are looking pretty good: we can get a list of commits in the release and a list of pull requests that might be in the release. Now, we want to filter that list of commits to remove items that are covered by a pull request. This is easy; we just compare the hashes and remove the matches.

Using the collection of commits for the latest release, we join the commits from the pull requests using the SHA hash and then select all release commits that have no matching commit in the pull requests2. However, we don't want to lose information just because we're losing noise, so we have to maintain a list of the pull requests that were matched so that we can build our release note history. To keep track, we will hold off on discarding any information by pairing up commits in the release with their prospective pull requests instead of just dropping them.

Going back to where we had a list of pull requests merged prior to our release, let us revisit getting the commits for those pull requests and this time, pairing them with the commits in the release to retain information.

Now we have a list of commits paired with their parent pull request, if there is one. Using this we can build a more meaningful set of changes for a release. If I run this on the latest release of the Octokit.NET repository and then group the commits by their paired pull request, I can see that the original list of 135 commits would be reduced to just 58 if each commit that belonged to a pull request were bundled into just one entry.

Next, we need to process the commits to remove those representing merges and other noise. These are things to discuss in the next post of this series where perhaps we will take stock and see whether this effort has been valuable in producing more meaningful release note generation. Until then, thanks for reading and don't forget to leave a comment.


  1. often changes are merged forward from one branch to another, especially if there are multiple release branches to support patch development and such 

  2. The join in this example is an outer join; we are taking the join results and using DefaultIfEmpty() to supply an empty collection when there was nothing to join 

Octokit and the Content of Releases

I started out my series on Octokit by defining a goal; to use GitHub repository history to build a basic summary of changes contained in a release. In order to do this, we need to define what a release is and then determine how we get the pertinent information to say what changes that release contains.

At a basic level, a release is a tagged point in the git repository. GitHub takes this one step further by making a release a first class concept as a lightweight git tag with additional attributes like a title and release notes. Octokit even allows first class access to GitHub releases in a repository, like so:

Great! With a little extra code, we can determine which release was the latest and then get all the commits in that release.

In the above code, we use MoreLinq to get the most recent release and then request all the commits in the repository on the same branch as that release up until the date the release was created. We request these commits using a CommitRequest object that specifies the query parameters. In this case, we want all the commits until the date of the release for the tag on which the release was made1. Of course, this will include everything ever done in that branch since the beginning of time, which is a bit of information overload. What we really want are the commits since the previous release.

Now we have taken the releases and used their CreatedAt dates to determine the most recent two and used the previous release date to set the Since date in our request. However, this code still has a flaw; we never said what branch the releases should be from. For all we know, the most recent two releases are on entirely different branches. To fix that, we need to filter the releases to just the branch we want.

The highlighted line is where we filter on the appropriate branch (it took some investigation to discover that the TargetCommitish property of a release is its branch name). We now have just the commits for the release branch we care about between the most recent release and the one before it.

In the next post, we will look at reducing the noise in the commit history using pull requests. Until then, thank you for stopping by and don't forget to leave a comment.

 


  1. The Sha property of the CommitRequest can be either a commit hash or branch/tag name 

And so it goes

You may have noticed I have not posted in a while. We recently moved from Michigan to Texas and during that time, I let a few lesser commitments slide. That is not to say I do not value my blog, I merely value other aspects of my life more1. Now that we are settled and some of the more frantic aspects of the move are over with, I thought it appropriate to get posting again and began crafting my next entry in my series on Octokit. However, there is something more pressing that I have to share first. I want to tell you about someone very special.

In 2001, a few months after having graduated from university and moving to Cambridgeshire, my housemate, Adam, and I decided to check out the local pub2. It was on that first visit to the Red Lion in Stretham that I met Mary, who at the time was working behind the bar. She was joyful, sparkling, kind, and funny. Like the most excellent of those who work a bar, she made us feel welcome, like we belonged. For the first time, I felt like Stretham was home.

The next time I remember seeing Mary was a day or so later when Adam and I were walking across the village green. She came walking towards us, holding the hand of a little girl.

Adam memorably said, “Is that yours?”

“That” turned out to be Mary’s daughter, Jordan. It also turned out that Mary, along with her adorably cheeky daughter, lived next door to us and over the months to follow we became friends. Most Thursdays3, Mary held her “Top of the P, Top of the I” club4 where we would share a drink, a smoke, and a lot of laughs, often while watching “Enders”5 or some other nonsense. I have many fond memories of us sitting in her lounge, kitchen, or backyard, in the pub, or in the beer garden behind it; all of them with Mary smiling and laughing and sparkling.

Mary and Chrissy

When I was happy, she would laugh with me. When I was sad, she would sit with me. When I was stupid, she would tell me. Mary became the best of friends; unafraid to be honest, never judging, always supportive. A counsel and a partner in crime (I suspect this is the case for many of her friends). On the day I left for the US, it was Mary that stood in her dressing gown in the backyard of her house to wave goodbye, smiling and sparkling.

On return trips to England, I always did what I could to get to Stretham and see all my friends, stopping by the Red Lion for far too many drinks and never enough good times. I did not always succeed. For those that live far from their friends and family, it is an all too familiar experience to never have enough time to see everyone. On one occasion I visited Cambridgeshire but could not see Mary, she understood.

“Next time,” she said.

And so it was that earlier this year, Chrissy and I stopped by Stretham to see Mary and Jordan. Though we spent some time at the Red Lion catching up with some old familiar faces, it was back at Mary’s I remember most. There we met the amazing young woman Jordan grew up to be, we shared stories of the times we had shared before6, and we got to know Russ, the love of Mary’s life. We spent as much time with them as they could stand and it was wonderful. Jordan was sarcastic and sassy, Russ was witty and wonderful, and Mary was smiling and sparkling, more than I ever remember her doing before. There was even one surviving PEPSI glass from the “Top of the P, Top of the I” club and we put it to good use. The time we spent with Mary and her family, seeing her happier than ever, surrounded by love was one of the highlights of our trip.

Mary and Family

"It takes a minute to find a special person, an hour to appreciate them, and a day to love them, but it takes an entire lifetime to forget them."

And so it goes. Yesterday, a dear friend reached out to me and informed me that Mary had died. Some time, while I was asleep or doing something else unremarkable, the world lost some of its shine. No reason. No fanfare. No sparkle.

Russ, Jordan, and the rest of Mary’s family and friends are grieving and I with them. There’s nothing more to say about that.

Every day of our lives, we carry our friends with us, no matter where they are. They are there when we cry and when we laugh, when we have to make difficult decisions, and when we just want to reminisce. I am grateful for the moments shared with my friends and for them making me a part of their world. Mary was one of a kind and everyone that knew her is better for it.


  1. like food, shelter, and love 

  2. I do not remember why we had not gone there sooner, nor the impetus that led to us going for the first time, though I dearly wish I could 

  3. I’m pretty sure it was Thursdays…my memory fails a little to be certain 

  4. Named after Mary’s PEPSI glasses, that had letters on the side making convenient measures for the mix of Bacardi and cola that we drank 

  5. EastEnders 

  6. like when Chrissy and Mary held me down while an 8 year old Jordan bound my hands with Selotape for no good reason other than “just because”