Octokit and the Documentation Nightmare

Before I get into the meat of this series of posts, I would like to set the scene. Like many organisations that perform some level of software development these days, we use GitHub. Here at CareEvolution, some developers use the web interface extensively, some use the command line, and others use the GitHub desktop client1, but most use a combination of two or more, depending on the task. This works great for developers, who have each found a comfortable workflow for getting things done, but it is not so great for those involved with DevOps, QA, or documentation where there is a need to find out user-friendly details of what the developers did. Quite often, a feature or bug fix involves several commits and while each has a comment or two, and perhaps an associated pull request (PR) or issue has a general description, but there is no definitive list of "this is what release X contains" that can be presented to a customer. Not only that but sometimes a PR or issue is resolved in an earlier release and merged forward. While we have lists of what a release is going to include, quite often there is more detail that we would like to include, and we often have additional changes as we adapt to the changing requirements of our customers. All this means that one or more people end up trawling the commits, trying to determine what the changes are. It is not a happy task.

"There is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things."

Niccolo Machiavelli
The Prince (1532)

Now, I know that this could all be avoided if people documented changes more clearly, perhaps added release notes to commits, raised issues for documentation changes, or created release notes on the release when it is made. However, no matter how noble change may be, anyone who has worked in process definition for any length of time will know that changing the behaviour of people is the hardest task of all, and therefore it should be avoided unless absolutely necessary. It was with that in mind that I decided mining the existing data for information would be an easier first step than jumping straight to asking people to change. So, with the aim of making life a little easier, I started looking at ways to automate the trawling.

I figured that by throwing out noisy and typical developer non-descriptive commits like "fixed spelling" or "updated comment", and by combining commits under the corresponding PR or issue, I could create useful summary of changes. This would not be customer-ready, but it would be ready for someone to turn into a release note without needing to trawl git history. In fact, if I included details of who committed the changes, it might even provide a feedback loop that would improve the quality of developer commit messages; developers do not like interruptions, so anyone asking for more detail on a commit they made should start to reinforce that if they wrote better commits, PRs, issues, they would get less interruptions.

Octokitty2

Octokit .NET logoAfter a dismissing using git locally to perform this task (I figured those who might need this tool would probably not want to get the repository locally) and reading up on the GitHub API a little, I cracked open LINQPad —my tool of choice for hacking— and went looking for a Nuget package to help. It was during that search that I happily stumbled on Octokit, the official GitHub library for interacting with the GitHub API. At the time of writing, Octokit reflects the polyglot nature of GitHub users, providing variants for Ruby, .NET, and Objective C, as well as experimental versions for Python, and Go. I installed the Octokit Nuget package into LINQPad and started hacking (there is also a reactive version for IObservable fans).

Poking around the various objects, and reading some documentation on GitHub (Octokit is open source), I got a feel for how the library wrapped the APIs. Though, I had not yet got any code running, I was making progress. Confident that this would enable me to create the tool I wanted to create, I started writing some code to gather a list of releases for a specific repository and stumbled over my first hurdle; authentication. It turns out it is not quite as straight-forward as I thought (the days of username and password are quite rightly behind us3), and so, my adventure began.

And then…

This is a good place to stop for this week, I think. As the series progresses, I will be piecing together the various parts of my "release note guidance" tool and hopefully, end up with a .NET library to augment Octokit with some useful history mining functionality. Next time, we will take a look at authentication with Octokit (and there will be code).


  1. OSX and Windows variants 

  2. or, James Bond for kids 

  3. OK, that's a lie, but I want to encourage good behaviour 

Reputation Is No Substitute For Knowledge

Last week, I regrettably ventured back to answering questions on StackOverflow. The question that lured me back was this one:

Due to the general confusion over this operator, my answer, though correct, was down-voted and derided as entirely wrong. Worst of all, one of the main detractors had over 300k in reputation and, rather than try what I had suggested, spent their time telling me I was wrong as their own incorrect answer received all the up-votes. In the spirit of StackOverflow as I once knew it, I edited my answer and answered the comments, trying to clear up the confusion and get the question answered adequately. As my answer got down-voted, more incorrect answers got up-voted. However, eventually, I was able to convince my main detractor that my answer was correct. So they promptly deleted all evidence that they ever thought otherwise and, without attribution, edited their once incorrect, top-voted answer to be correct.

Though it stings a little1, I do not mind that my answer did not get accepted nor that it did not get the most votes; the question was answered correctly and that's the point of the site. What I find most disagreeable is the unsporting behavior that undermined the sense of community that once pervaded StackOverflow. I left the whole experience feeling like an outsider. In the past, those with wrong answers would delete theirs in favor of the right one, or they would edit theirs, but give credit to the right one. People would (in the most part) treat each other with respect and see reputation as a sign of being a good citizen, not necessarily a knowledgeable one. Not anymore.

I wish I could show the comments I received when answering this question, but they were deleted2. However, the general pattern of this and other experiences appears to be that someone with a high reputation score down-votes and derides other answers, then once the correct answer is clear, takes everything from the correct answers posted to edit into their own, which then earns all the reputation. It is an embittering experience that I know others have shared.

In the beginning, earning reputation and badges encouraged people to get things right and to help each other out. Now the site has matured, the easy questions are answered, and the gap between the newcomers and those with the highest reputation is huge. Newcomers languish in poverty with little opportunity, if any, to reach the top, while those at the top benefit from a bias toward answers and opinions that come from those with large reputation scores. What once incentivised good behavior and engagement, seems to have led to bullying and dishonesty. I am not saying that all people with high reputation engage in unscrupulous practices on StackOverflow —there are many generous and humble members of the community —unfortunately, bad experiences outweigh good experiences 5-1 (or as high as 12-1), so the actions of a few can poison the well.

The root of the problem as I see it3 is that reputation has become (or perhaps always was) over-valued, and in its pursuit, some have lost sight of what StackOverflow was trying to achieve; community. The community that made it special, that made me feel like I belonged, is gone, and reputation is no substitute for knowledge. What was once an all-for-one, one-for-all environment has, in the competition for reputation, turned toxic4.

I have no doubt that many reading this will think I am misrepresenting the situation, overreacting, or just plain wrong, and that is OK; I hope that those people are right, that this is not a trend, and that the overall community remains friendly and constructive. Personally, I will think twice before involving myself in answering (or even asking) a question on StackOverflow again.

Ultimately, StackOverflow works as long as the right answers get provided; but if those with the knowledge to answer get disillusioned and leave, from where will those right answers come?

Today's featured image is "Façade of the Celsus library, in Ephesus, near Selçuk, west Turkey" by Benh LIEU SONG. It is licensed under CC BY-SA 3.0. Other than being resized, the image has not been modified.


  1. we all like recognition for being right 

  2. I also deleted mine, since they were without context 

  3. if it is agreed that there is one 

  4. The fact that I even felt wronged may well be an indicator of that toxicity and my own part in its creation 

Testing AngularJS: Directives

So far in this series of AngularJS-related posts, we have looked at some utility factories for tracking web requests and preventing navigation requests, whether in-app or not, and how to write tests for those factories using Jasmine. In this post, we will look at how I generally test directives and how that impacts their structure.

Directives

Directives are the things we write in AngularJS that allow us to extend and modify the HTML language. We provide a template, some code-behind, and data binding, and AngularJS performs its magic to create a new behavior or control that is inserted via tag, attribute or CSS class. However, directives can at first seem difficult to test.

Let's look at a convoluted example.

In this example we have a very simple directive. It specifies a template, some scope, and a controller where the business logic lives. It is used like this:

As it stands, there are a one or two things that either make this difficult to test or (I feel) could be clearer1. Let's refactor this directive with testing in mind.

Tests

When I have a working directive that I want to test, I tend to start writing tests. I realise this seems obvious, but it is worth noting as I do not tend to do much refactoring before writing tests and I don't tend to write tests before I have something to test. I find it much more useful to get a concept working and then think about what tests should be there and how I can implement those tests. When I find friction in authoring those tests, I identify my refactoring priorities.

Considering the directive presented above, what tests do we need? Here is the list of things that I think should be tested:

  • That the directive exists
  • That the directive compiles
  • That the alert shows when the button is clicked
  • That the alert mentions the field name
  • That the alert mentions the field value

Testing that the directive exists is easy (note that we have to add Directive to the end for the directive to be injected).

Checking it compiles is a little more convoluted, but not difficult.

The linking function that $compile returns is called because  $compile can succeed even if our fixture has a typo.

Using knowledge of our directive template, we can verify that the linking worked without going so far as to validate every aspect of the magic Angular does for us. In fact, since this is really intended to test the code that gets called inside the directive when it is linked to a scope, the template for the directive could be as simple as <div></div> (we will take a closer look in a future post at how we can make the directive template replaceable).

Testing the Alert

Testing that the directive exists, compiles and links is relatively straightforward when compared with testing that the alert is shown and shows the right thing. In order to test the alert, we need to be able to invoke showField, the method that shows it. We also need to check that showField actually shows the alert as we would like.

To invoke showField, we might try using the scope that is passed to the link method returned from $compile. Since the method is added to the scope in the controller, this should work, right? Of course not; this directive has an isolate scope and as such the method has been added to its own copy of the passed scope. We can get to that isolate scope from the compiled element using the isolateScope() method;

…but I feel like such a test involves too much setup ceremony just to get to the test2 and it relies on too many things outside of just the bit we want to validate. Instead, what if we could get at the controller independently of the directive?

Now our controller is completely separate from the directive, which refers to the controller by name. This means we can write what I feel are clearer directive tests. Using a fake for the controller, we isolate the directive from the real thing and any side-effects it may have;

…and now the controller can have its own tests too;

$window

However, we are not quite finished. How do we know that the alert is shown when our showField method is called? We could spy on the global window object, but that's not really very Angular-y and our alert will get shown during testing (goodness knows what other side-effects we might face by using the global window object). What we need is an injectable version of window that we can replace with a fake during our tests. Not unsurprisingly, AngularJS has us covered with the $window service.

Now with our controller rewritten to use $window, we can tidy up and complete its test cases.

And there you have it. Some simple steps that make unit testing directives a little easier.

Finally…

In this post, we have taken a very brief look at how to structure a directive to simplify unit testing by separating the directive declaration from its controller and taking advantage of Angular services such as $window.

Although we have not covered some of the more complex directive concepts such as the link function and DOM manipulation3, these simple steps should take you a long way towards providing better test coverage of your AngularJS widgets.

Until next time, take care and don't forget to leave a comment.


  1. Not unexpected since I wrote it to discuss these things 

  2. we have to compile a fixture with a scope and then get the isolate scope from the element 

  3. I may cover that in a future post if there is interest or the whim takes me 

Testing Times

Developers, testers and testing

In the world of software, there are developers and there are testers. The developers often design and implement the software while the testers define and execute the test plans. Software engineering requires both testers and developers, and together they make quality software; one by finding problems and the other by solving problems1. At least, that's the way it should be. Unfortunately, many developers (including myself) have found themselves in situations where the QA department is nonexistent, where testing and the associated test plan updates lurk at the end of every development cycle or feature implementation.

System testJust to be clear, we're not talking unit tests like those used in test-driven development (TDD) with frameworks like NUnit or MSTest. Unit tests and TDD are somewhat unique in that they take the developer's strength of solving problems and trickpersuade developer's into seeing testing as yet another problem in need of resolution (just how do you prove a requirement was met – to the TDD Cave, Codeman!).

Sadly, manual tests found in system testing, integration testing and regression testing are not so exciting. They don't usually present cunning problems to be solved but instead provide a means for mind-numbing hours following detailed, inane instructions where the result feels obvious and the rewards are few. At least, that's my experience as a developer performing tests; the same cannot be said of testers. I've worked with some very talented, passionate quality assurance professionals whose joy found in their craft was inspiring and of whom I have been envious when I too have found myself burdened by testing.

Finding those team mates who take pride in testing and making a product better is like striking gold, but even those that find schadenfreude in identifying a colleague's mistakes can be a better option to a developer than having to run the testing themselves. However, dedicated resources for quality assurance are often seen as a luxury2, leaving developers with little option but to take that responsibility on themselves.

To be clear, I'm trying to say that developers generally hate testing and more specifically, I hate testing, but we'll do it anyway if pushed.

WHHHHAAAT??!

At this point you may be surprised to discover that I recently found myself testing some software. Whether it was a poorly defined test, a flaky feature, or just the mundanity of repeating the same operations (albeit with subtle adjustments) over and over and over again, it left me frustrated, weary and disengaged. Testing is just not my thing, but I do it because I have to – releasing untested software should never be an option for a professional software developer; our users are not our QA department. The all too familiar experience reminded me of steps that developers can take when they're the ones that have to update and execute manual testing; steps that I've seen in action and that make testing almost pleasurable (almost).

Just update the test plan

Have you ever updated a test plan without checking the test was correct, or perhaps executed a test plan that was incorrect? Updating a test plan is tedious, we have to check that existing tests are still relevant and work out where there are gaps in the test coverage. This usually means looking at requirements documents and change requests and determining various test paths, expected results, etc. It can be a lot of work and it is all too easy to fall into the trap of skipping some steps, like validating the test is correctly defined or pretending that there's no way the existing plan missed something. Not only that, but if you've diligently updated the test plan, validating each test as you go, executing it all over again is even more painful because you already know what does and does not work from updating the tests in the first place.

So, do it once and do it right. If you carefully update the test plan, validating existing tests, updating others and creating new ones, you will find yourself testing the product anyway. As tests that should work don't, change requests will get raised and the product will improve. Not only that, but you'll only need to update the document once and you won't need to run the tests more than is absolutely necessary. To cap it off, the act of defining tests is pretty close to problem solving, making it a little less tedious for a developer to perform (though it is documentation, so, you know, don't hurt yourself or anything).

Assume the tester knows nothing (and is a little slow)I met a hawk and it was red

All too often, I come across test plans that are written like a kindergarten story.

Start the application. And then open a file. And then click OK. And then check the background is white and the caption says "Bite me!".

Paragraphs of simple instructions, often with steps missing that the author assumes the tester will know and without any explanation of what it means if that test fails. Instead of this mess, introduce each test with an overview of its purpose and what failure means, followed by test instructions each on a separate line. This not only helps you and your team mates when running the tests but it also helps when they come to update the test plan. Think of the test as code; you wouldn't expect the processor to guess when you miss out lines of code (I hope) so don't expect a tester to do the same; don't forget to add comments where more detail is needed (such as why it's important to change what locale the system is using); and number each step so that it can be referred to easily in notes and change requests, e.g. "Test 2.6, step 10 failed with a value of 20 where 21 was expected"3. If you do this, you will thank yourself later.

Provide context for the results

When performing the test, you will want to be recording results for each step. When reviewing results, you will usually want to see the test step that garnered them, especially if there is a failure or an ambiguous result. Save yourself some time by specifying your tests as a table with a column for results. That way, results are recorded next to the test definition making both recording and reviewing much easier. Not only that, but you don't need to maintain a results sheet and the test definitions separately or contend with different people recording the results in different formats.

Conclusions

If you follow these three simple steps, you should end up with test definitions that look less like an account of your weeks at summer camp when you were 7 and more like the example below.

This test checks the flange sprocket exposes the doobrey flap.

StepInstructionsResults
10Open the flange sprocket. You should see the flange sprocket open.Pass – opened
20Press the doobrey flap.Fail – unable to locate doobrey flap. Test lacking sufficient detail or doobrey flap was not exposed.
30

Of course, all this assumes you don't have a QA team or team members (or even some tools that help you define and execute manual testing). If you do, that's great; respect your QA team members (or your tools) and the work they do to keep your users from deploying their wrath upon thee. For the rest of us, stuck with ourselves and our office productivity applications in which to define and record our testing, following these tips will make our testing life (and that of those around us) just that little bit less tedious. Who knows, some of you might even start enjoying it.


  1. This is a very simplistic overview, I know.  

  2. There are valid and not so valid reasons for this, but we're not going to get into that here. 

  3. You might also consider spacing step numbers by 10 so it's easier to insert additional steps without renumbering all subsequent steps.