The Need For Speed

Hopefully, those who are regular visitors to this blog¹ have noticed a little speed boost of late. That is because I recently spent several days overhauling the appearance and performance with the intent of making the blog less frustrating and a little more professional. However, the outcome of my effort turned out to have other pleasant side effects.

I approached the performance issues as I would when developing software; I used data. In fact, it was data that drove me to look at it in the first place. Like many websites, this site uses Google Analytics, which allows me to poke around the usage of my site, see which of the many topics I have covered are of interest to people, what search terms bring people here (assuming people allow their search terms to be shared), and how the site is performing on various platforms and browsers. One day I happened to notice that my page load speeds, especially on mobile platforms, were pretty bad and that there appeared to be a direct correlation between the speed of pages loading and the likelihood that a visitor to the site would view more than one page before leaving² . Thankfully, Google provides via their free PageSpeed Insights product, tips on how to improve the site. Armed with these tips, I set out to improve things.

Now, in hindsight, I wish I had been far more methodical and documented every step— it would have made for a great little series of blog entries or at least improved this one —but I did not, so instead, I want to summarise some of the tasks I undertook. Hopefully, this will be a useful overview for others who want to tackle performance on their own sites. The main changes I made can be organized into server configuration, site configuration, and content.

The simplest to resolve from a technical perspective was content, although it remains the last one to be completed mainly due to the time involved. It turns out that I got a little lazy when writing some of my original posts and did not compress images as much as I probably should have. The larger an image file is, the longer it takes to download, and this is only amplified by less powerful mobile devices. For new posts, I have been resolving this as I go by using a tool called PNGGauntlet to compress my images as either JPEG or PNG before uploading them to the site. Sadly, for images already uploaded to the site, I could only find plugins that ran on Apache (my installation of WordPress is on IIS for reasons that I might go into another time), would cost a small fortune to process all the images, or had reviews that implied the plugin might work great or might just corrupt my entire blog. I decided that for now, to leave things as they are and update images manually when I get the opportunity. This means, unfortunately, it will take a while. Thankfully, the server configuration options helped me out a little.

On the server side, there were two things that helped. The first, to ensure that the server compressed content before sending it to the web browser, did not help with the images, but it did greatly reduce the size of the various text files (HTML, CSS, and JavaScript) that get downloaded to render the site. However, the second change made a huge difference for repeat visitors. This was to make sure that the server told the browser how long it could cache content for before it needed to be downloaded again. Doing this ensured that repeat visitors to the site would not need to download all the CSS, JS, images, and other assets on every visit.

With the content and the server configuration modified to improve performance, the next and most important focus was the WordPress site itself. The biggest change was to introduce caching. WordPress generates HTML from PHP code. This takes time, so by caching the HTML it produces, the speed at which pages are available for visitors is greatly increased. A lot of caching solutions for WordPress are developed with Apache deployments in mind. Thankfully, I found that with some special IIS-specific tweaking, WP Super Cache works great³ .

At this point, the site was noticeably quicker and almost all the PageSpeed issues were eliminated. To finish off the rest, I added a few plugins and got rid of one as well. I used the Autoptimize plugin to concatenate, minify, compress, and perform other magic on the HTML, CSS, and JS files (this improved download times just a touch more by reducing the number of files the browser must request, and reducing the size of those files), I added JavaScript to Footer, a plugin that moves JavaScript to after the fold so that the content appears before the JavaScript is loaded, I updated the ad code (from Google) to use their latest asynchronous version, and I removed the social media plugin I was using, which was not only causing poor performance but was also doing some nasty things with cookies.

Along this journey of optimizing my site, I also took the opportunity to tidy up the layout, audit the cookies that are used, improve the way advertisers can target my ads, and add a sitemap generator to improve some of the ways Google (and other search engines) can crawl the site⁴. In all, it took about five days to get everything up and running in my spare time.

So, was it worth it?

From my perspective, it was definitely worth it (please let me know your perspective in the comments). The image above shows the average page load, server response, and page download times before the changes (from January through April – top row) and after the changes (June – bottom row). While the page download time has only decreased slightly, the other changes show a large improvement. Though I cannot tell for certain what changes were specifically responsible (nor what role, if any, the posts I have been writing have played⁵ ), I have not only seen the speed improve, but I have also seen roughly a 50-70% increase in visitors (especially from Russia, for some reason), a three-fold increase in ad revenue⁶, and a small decrease in Bounce Rate, among other changes.

I highly recommend taking the time to look at performance for your own blog. While there are still things that, if addressed, could improve mine (such as hosting on a dedicated server), and there are some things PageSpeed suggested to fix that are outside of my control, I am very pleased with where I am right now. As so many times in my life before, this has led me to the inevitable thought, "what if I had done this sooner?"

hopefully, there are regular visitors [↩]
The percentage of visitors that leave after viewing only one page is known as the Bounce Rate [↩]
Provided you don't do things like enable compressing in WP Super Cache and IIS at the same time, for example. This took me a while to understand but the browser is only going to strip away one layer of that compression, so all it sees is garbled nonsense. [↩]
Some of these things I might blog about another time if there is interest (the cookie audit was an interesting journey of its own). [↩]
though I possibly could with some deeper use of Google Analytics [↩]
If that is sustained, I will be able to pay for the hosting of my blog from ad revenue for the first time [↩]

Debugging IIS Express website from a HyperV Virtual Machine

Recently, I had to investigate a performance bug on a website when using Internet Explorer 8. Although we are fortunate to have access to BrowserStack for testing, I have not found it particularly efficient for performance investigations, so instead I used an HyperV virtual machine (VM) from modern.IE.

I had started the site under test from Visual Studio 2013 using IIS Express. Unfortunately, HyperV VMs are not able to see such a site out-of-the-box. Three things must be reconfigured first: the VM network adapter, the Windows Firewall of the host machine, and IIS Express.

HyperV VM Network Adapter

In HyperV, select Virtual Switch Manager… from the Actions list on the right-hand side. In the dialog that appears, select New virtual network switch on the left, then Internal on the right, then click Create Virtual Switch. This creates a virtual network switch that allows your VM to see your local machine and vice versa. You can then name the switch anything you want; I called mine LocalDebugNet.

To ensure the VM uses the newly created virtual switch, select the VM and choose Settings… (either from the context menu or the lower-right pane). Choose Add Hardware in the left-hand pane and add a new Network Adapter, then drop down the virtual switch list on the right, choose the switch you created earlier, and click OK to accept the changes and close the dialog.

Now the VM is setup and should be able to see its host machine on its network. Unfortunately, it still cannot see the website under test. Next, we have to configure IIS Express.

IIS Express

Open up a command prompt on your machine (the host machine, not the VM) and run ipconfig /all . Look in the output for the virtual switch that you created earlier and write down the corresponding IP address¹.

Open the IIS Express applicationhost.config file in your favourite text editor. This file is usually found under your user profile.

notepad %USERPROFILE%\documents\iisexpress\config\applicationhost.config

Find the website that you are testing and add a binding for the IP address you wrote down earlier and the port that the site is running on. You can usually just copy the localhost binding and change localhost to the IP address or your machine name.

You will also need to run this command as an administrator to add an http access rule, where <ipaddress> should be replaced with the IP you wrote down or your machine name, and <port> should be replaced with the port on which IIS Express hosts your website.

netsh http add urlacl url=http://<ipaddress>:<port>/ user=everyone

At this point, you might be in luck. Try restarting IIS Express and navigating to your site from inside the HyperV VM. If it works, you are all set; if not, you will need to add a rule to the Windows Firewall (or whatever firewall software you have running).

Windows Firewall

The VM can see your machine and IIS Express is binding to the appropriate IP address and port, but the firewall is preventing traffic on that port. To fix this, we can add an inbound firewall rule. To do this, open up Windows Firewall from Control Panel and click Advanced Settings or search Windows for Windows Firewall with Advanced Security and launch that.

Select Inbound Rules on the left, then New Rule… on the right and set up a new rule to allow connections the port where your site is hosted by IIS Express. I have shown an example here in the following screen grabs, but use your own discretion and make sure not to give too much access to your machine.

Once you have set up a rule to allow access via the appropriate port, you should be able to see your IIS Express hosted site from inside your VM of choice.

As always, if you have any feedback, please leave a comment.

You can also try using the name of your machine for the following steps instead of the IP [↩]

CiviCRM deployment on IIS WordPress

At Ann Arbor Give Camp this year, I worked on a team looking into donation management options for non-profits. Thanks to Dr. Milastname (it would be inappropriate to reveal his true identity), we found CiviCRM and spent much of the weekend getting familiar with its deployment and functionality inside of WordPress. CiviCRM integration with WordPress is a relatively new feature, so it was not totally unsurprising that we encountered one or two issues. The first and by far the biggest problem we encountered was the White Screen of Death (WSOD).

After some debugging (which involved editing a couple of PHP files inside of the WordPress and CiviCRM systems), we discovered that a PHP add-in used by CiviCRM for templating was relying on the `open_basedir` variable and this was not set on our IIS-based system. This caused the templating add-in to fail, halting the rendering of the CiviCRM admin screen and resulting in the WSOD.

To rectify this problem, I edited `wp_config.php` to introduce the `open_basedir` variable just before the `require` statement for `wp_settings.php`. I set the variable to the path of the WordPress deployment (`ABSPATH`) and refreshed the CiviCRM admin screen.

ini_set('open_basedir', ABSPATH);

This fixed the WSOD and enabled us to continue our evaluation of CiviCRM¹. We also raised an issue against the CiviCRM project and added a post to the CiviCRM forms, ensuring the lessons we learned would benefit future users of CiviCRM.

And discover a bug that I had introduced all by myself [↩]

Drop the BOM: A Case Study of JSON Corruption in WordPress

In September, I attended Ann Arbor Give Camp, a local event that connects non-profits with the local developer community to fulfill technological goals. As part of the project I was working on, I installed a plugin called CiviCRM into a WordPress deployment that was running on an IIS-based server.

It turned out that WordPress integration for CiviCRM was relatively new and a problem unique to IIS-based deployments existed after installation. This led to a white screen when I tried to access CiviCRM. I spent some time troubleshooting and eventually found the issue after I edited two files to track it down. The fix was quickly implemented. Unfortunately, I then discovered that some other features were not working properly.

The primary places this new issue surfaced were in displaying dialog windows within CiviCRM. It turned out that these dialogs obtained their UI via an AJAX call that returned some JSON and for some reason, jQuery was indicating that the call failed. Investigating further, I saw that the API call was successful (it returned a 200 status result) and the JSON appeared completely fine. How strange.

I made some debug changes to the JavaScript using the Google Chrome development tools and looked at the failure method jQuery was calling. In doing so, I discovered jQuery was reporting a parsing error for the JSON result. This seemed bizarre, after all, the JSON looked fine to me. I decided to verify it by copying and pasting it into Sublime. Still, the JSON looked just fine. Being tenacious, I saved the JSON to a text file and then opened it in Visual Studio's binary editor and there, the problem appeared. There were two characters at the start of the file before the first brace: byte order marks.

Corrupted JSON in Google Chrome developer tools

A byte order mark (often referred to as a BOM) is a Unicode character used to indicate the endianness (byte order) of a text file or stream¹. JSON is not supposed to include them at all. In hindsight, I could have seen this issue much sooner if I had paid closer attention to the JSON response in the Network tab of Chrome's developer tools. This view had shown two red dots (see above) before the opening brace, each dot corresponding to a BOM that Chrome knew shouldn't be there. Of course, I had no idea what they meant and so I promptly ignored them. Lesson learned.

So, armed with the knowledge of why the JSON was causing parser errors, I had to find out what was causing this malformation and fix it. After reading about how a BOM in an incorrectly formatted PHP file² could cause BOMs to be prepended in the PHP output, I started looking at each PHP file that would be touched when generating the API response. Alas, nothing initially stood out. I was getting frustrated when I had an epiphany; I had edited exactly two files in trying to fix the installation issue and there were exactly two BOMs. Coincidence?

I went to the two files that I had edited, downloaded them and discovered they both had BOMs. I re-saved them, this time without a BOM and uploaded them back to the site, which fixed the JSON corruption and got the CiviCRM plug-in in to working order.

In tracking down and fixing this self-made issue, I learned a few valuable lessons:

Learn to use my developer tools
Never assume it is not my fault
It pays to understand how things work

Hopefully, my misfortune in this one incident will help someone track down their own issue with corrupted JSON in WordPress. If so, please share in the comments. Together, our mistakes can be someone else's salvation.

Wikipedia – http://en.wikipedia.org/wiki/Byte_order_mark [↩]
one saved as Unicode with byte order mark [↩]