Octokit and the Documentation Nightmare

Before I get into the meat of this series of posts, I would like to set the scene. Like many organisations that perform some level of software development these days, we use GitHub. Here at CareEvolution, some developers use the web interface extensively, some use the command line, and others use the GitHub desktop client¹, but most use a combination of two or more, depending on the task. This works great for developers, who have each found a comfortable workflow for getting things done, but it is not so great for those involved with DevOps, QA, or documentation where there is a need to find out user-friendly details of what the developers did. Quite often, a feature or bug fix involves several commits and while each has a comment or two, and perhaps an associated pull request (PR) or issue has a general description, but there is no definitive list of "this is what release X contains" that can be presented to a customer. Not only that but sometimes a PR or issue is resolved in an earlier release and merged forward. While we have lists of what a release is going to include, quite often there is more detail that we would like to include, and we often have additional changes as we adapt to the changing requirements of our customers. All this means that one or more people end up trawling the commits, trying to determine what the changes are. It is not a happy task.

"There is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things."

Niccolo Machiavelli
The Prince (1532)

Now, I know that this could all be avoided if people documented changes more clearly, perhaps added release notes to commits, raised issues for documentation changes, or created release notes on the release when it is made. However, no matter how noble change may be, anyone who has worked in process definition for any length of time will know that changing the behaviour of people is the hardest task of all, and therefore it should be avoided unless absolutely necessary. It was with that in mind that I decided mining the existing data for information would be an easier first step than jumping straight to asking people to change. So, with the aim of making life a little easier, I started looking at ways to automate the trawling.

I figured that by throwing out noisy and typical developer non-descriptive commits like "fixed spelling" or "updated comment", and by combining commits under the corresponding PR or issue, I could create useful summary of changes. This would not be customer-ready, but it would be ready for someone to turn into a release note without needing to trawl git history. In fact, if I included details of who committed the changes, it might even provide a feedback loop that would improve the quality of developer commit messages; developers do not like interruptions, so anyone asking for more detail on a commit they made should start to reinforce that if they wrote better commits, PRs, issues, they would get less interruptions.

Octokitty²

After a dismissing using git locally to perform this task (I figured those who might need this tool would probably not want to get the repository locally) and reading up on the GitHub API a little, I cracked open LINQPad —my tool of choice for hacking— and went looking for a Nuget package to help. It was during that search that I happily stumbled on Octokit, the official GitHub library for interacting with the GitHub API. At the time of writing, Octokit reflects the polyglot nature of GitHub users, providing variants for Ruby, .NET, and Objective C, as well as experimental versions for Python, and Go. I installed the Octokit Nuget package into LINQPad and started hacking (there is also a reactive version for `IObservable` fans).

Poking around the various objects, and reading some documentation on GitHub (Octokit is open source), I got a feel for how the library wrapped the APIs. Though, I had not yet got any code running, I was making progress. Confident that this would enable me to create the tool I wanted to create, I started writing some code to gather a list of releases for a specific repository and stumbled over my first hurdle; authentication. It turns out it is not quite as straight-forward as I thought (the days of username and password are quite rightly behind us³), and so, my adventure began.

And then…

This is a good place to stop for this week, I think. As the series progresses, I will be piecing together the various parts of my "release note guidance" tool and hopefully, end up with a .NET library to augment Octokit with some useful history mining functionality. Next time, we will take a look at authentication with Octokit (and there will be code).

OSX and Windows variants [↩]
or, James Bond for kids [↩]
OK, that's a lie, but I want to encourage good behaviour [↩]

Corporate Dogma: Good Intentions, Bad Policies

Recently, a colleague wanted to update our corporate profile on a well-known website for sharing interview experiences; we will call them Brickwindow. When signing up for a free corporate account, he filled out the fields honestly, stating his name, his job title, and checking the box that stated he represented HR, which he does as part of his various duties. 24 hours later (or thereabouts), the response came back saying that Brickwindow were not permitting him to open the account as his job title wasn't HR-related. They would need a C-level employee or member of the HR department to verify his authority (or to open the account themselves).

Here's the issue. Our organization does not have an HR department or traditional C-level employees like CEO, CFO or CTO. Just as we are trusted to make good software engineering decisions, we are trusted to make good decisions in other aspects of the business, including who we hire. There are no managers, no Chief Gubbins Officers, and no HR departments. So, my colleague wrote back to Brickwindow outlining how our company structure just does not fit their rules. He assured them that he is authorized to make HR decisions. Shortly thereafter, they responded reiterating that he could not have the account and asserting that they are committed to the security of their users.

So, I signed up.

I have the same job title as my colleague, but since we can choose our own job titles, I temporarily promoted myself to Assistant to the Director of HR. After submitting my application, I pondered on how they might verify this and quickly updated my LinkedIn profile to state the same title¹. About thirty minutes later I had an account. It was that easy. My colleague had just been too honest. As for the commitment to user security, Brickwindow sent me my new user name and password in plain text. Brilliant². I quickly changed it and then forwarded the account details to my colleague so he could continue with the task he had attempted to do the day before.

I think this tale serves as a great example of how we can get bogged down in process and miss the purpose of an activity altogether. In trying to make sure that no one could just create an account for any old company, Brickwindow made assumptions about its corporate user base that meant they could not adapt when faced with something that did not conform. Not only that, but they were so busy trying to enforce those assumptions, they missed the glaring loopholes. All they managed to achieve was a huge waste of time; theirs and ours. This had been an opportunity for Brickwindow to demonstrate they are forward thinking and adaptable, but instead they made it clear they are stuck in the past with the monolithic corporate structures of C-level employees and middle management. In the end, they failed on three major points: identity, roles, and security. We only persevered because we appreciate the primary service provided by Brickwindow.

So, next time you are faced with a situation that does not fit the script, consider whether the script needs to change before trying to change the situation. These conflicts are an opportunity to impress, not stick to rigid rules that don't even achieve the goals they intend to.

If you have any similar tales of well-intentioned processes gone wrong, or corporate dogma that gets in the way, please post them in the comments.

As for me, I have since demoted myself back to a Senior UX Engineer (a title I had chosen when I found out I would be speaking at CodeMash 2.0.1.4). It turns out HR was just not a good fit for me.

Apologies to anyone who was confused [↩]
sarcasm [↩]

Change Requests

Much like my previous post on Meeting Etiquette, this is a topic I feel strongly about. I am sure there are good reasons that people will hate some of my suggestions and I'd love to hear them, but here are my views on change requests¹ based on my personal experiences.

All work is a change

I loathe projects that differentiate new work from changes to existing work. It creates two different process flows for little gain, creating points for confusion and mistakes. If all work items, whether new features, bug fixes or enhancements to existing features are raised as change requests, the work flow is the same. Everything should be tied back to requirements, regardless of the type of work, so arguments that claim there is a difference just don't wash with me. Consider a new feature as a change from not having it to having it, after all, that's exactly what it is.

Whatever the work is that is being conducted must still be implemented, reviewed and tested against requirements. Why make it harder than it needs to be?

Specify requirements, not solutions

There are many times I've been assigned a requirement that tells me how to fix something, not what needs fixing. Let's face it, everyone has an opinion but change requests are not the place to express them (except perhaps as a suggestion in the comments somewhere). A change request should clearly state the requirements that drive the change (i.e. the things that can be used to identify when the change request has been resolved) and any other information that may help (for example, steps to reproduce a bug or some rationale behind the change required).

Be descriptive

If I see one more change request with a summary or title like "Change to menu dropdown" or "Display control update", I will be rather miffed and may hurt someone (I'm British, "miffed" is just above "peeved" on the British Standards Anger Scale²). The title of a change request is very important and should give a clear indication of what the change request actually requires. Think of it a bit like twitter; it's much nicer reading some useful information in a tweet than it is to learn that someone just had a coffee. If the title is not clear, time is wasted in going to look at the description every time someone sees that change request. Every status meeting, every discussion, click click click. Save everyone the effort and get it right first time, and if you spot a title that isn't clear enough, fix it right away.

Add value

Finally, when adding comments, additional description, attachments or anything else to a change request, make sure it adds value. Leave an trail for those who follow in your footsteps so that they can discover what changed and why. Document important discussions and decisions. If you don't, you are destined to go around in circles.

Manage releases by managing change

Target changes at releases and review new changes regularly. This way, new requests raised during that release cycle can be considered for inclusion and deferred changes can be ignored until after the release. Each time a new release is started, review all the open requests and determine if they should be rejected, deferred or included in that release. Justify and document rejections in case a duplicate is raised and make sure to link duplicate issues as they can add value to one another.

Have meaningful states

I feel that there are the following possible states for a change request in any sane process to manage them.

Raised
Assigned
In progress
Ready for review
Passed review
Merged to trunk
Rejected
Closed

These are clear, unambiguous states.

If something is marked as "Raised", it hasn't been assigned to any release and no work should be happening on it.
If it's "Assigned", it should be targetted at a release (even if it's only intended for investigation at first – it can always be removed from the release back to "Raised" or rejected and closed).
If someone is working on something, that something should be marked as "In progress" as this helps to track progress at a glance and can also be useful if resources become available and things need reassigning.
If something passes testing, close it, unless you really don't trust the test team, in which case have a "Passed test" state and then review the results before closing³.

Considering these basic states, the workflow looks something like this:

You could add additional states if you so desired but I feel that these cover the bases well enough and provide an easy to follow work flow.

Closed means closed

Change request management gets messy when the process allows for closed requests to be re-opened. If a closed request seems like it really is now needed, raise a new request. Don't close change requests just because they aren't being done right away; if it's a real issue, then it should remain open until it is resolved.

Not always, but mostly

The guiding principle for me when it comes to change requests is simplicity. Don't make your process more complicated than it needs to be. Make it easy to follow and hard to get wrong. While I'm yet to encounter a project that required anything different to what I've suggested, I am certain there are exceptions, so if you have any, let me know. This is often a contentious, polarising topic, so I expect someone, somewhere to emphatically insist I am wrong. I can accept that, so I look forward to finding out even better ways to manage change.

I made that up [↩]
You may also know them as issues, bugs, defects or some other moniker that ultimately means "a repository of things to do". [↩]
Flippant remarks aside, this may be valuable if you need to perform a round of customer acceptance testing after internal testing before closing out change requests. [↩]

Octokitty2