🙇🏻‍♂️ Introducing ancesdir

Photo by Maksym Kaharlytskyi on Unsplash

After many years of software development, I finally published my own NPM package. In fact, I published two. I was working on my checksync tool when I realised that I needed the package that this blog introduces. More on checksync in the next entry.

https://www.npmjs.com/package/ancesdir

🤔 What is root? Where is root?

Quite often, when working on some projects at Khan Academy, we need to know the root directory of the project. This enables us to write tools, linters, and tests that use root-relative paths, which in turn can make it much easier to refactor code. However, determining the root path of a project is not necessarily simple.

First, there is working out what identifies the root of a project. Is it the node_modules directory? The package.json file? The existence of .git folder? It may seem obvious to use one of these, but all these things have something in common; they don't necessarily exist. We can configure our package manager to have package.json and node_modules in non-standard places and we might change our source control, or not even run our code from within a clone of our repository. Determining the root folder by relying on any of these things as a marker is potentially not going to work.

Second, the code to walk the directory structure to find the given "marker" file or directory is not trivial. Sharing a common implementation within your project means everything that needs it, needs to locate it; in JavaScript, that means a relative path, at which point, you may as well just use a relative path to the known root directory and skip the shared approach all together. Yet, if you don't share a common implementation from a single location, then the code has to be duplicated everywhere you need it. I don't know about you, but that feels wrong.

💁🏻‍♂️ Solution: ancesdir

The issue of sharing a common implementation is easiest to solve. If that common implementation is installed as an NPM package, we don't need to include it via a relative path; we can just import it by its package name. There are packages out there that do this, but the ones I found all assumed some level of default setup, failing to acknowledge that this may change. In turn, they did not support a monorepo setup where there could be multiple sub-projects. How could one find the root folder of the monorepo from within a sub-project if all we used to identify the root folder were package.json? What if we wanted to sometimes get the root of the sub-project and sometimes the root of the monorepo?

I needed a way to identify a specific ancestor directory based on a known marker file or directory that would work even with non-standard setups. At Khan Academy, we have a marker file at the root of the project that is there solely to identify its parent directory as the project root. This file is agnostic of tech stack; it's just an empty file. It is solely there to say "this directory is the root directory". No tooling changes are going to render this mechanism broken unexpectedly unless they happen to use the same filename, which is unlikely. This way, we can find the repository root easily by locating that file. I wanted a package that could work just as easily with this custom marker file as it could with package.json.

I created ancesdir to fulfill these requirements1.

yarn add ancesdir

The API is simple. In the default case, all you need to do is:

import ancesdir from "ancesdir";

console.log(`ancesdir's root directory is ${ancesdir()}`);

If you have a standard setup, with a package.json file, you will get the ancestor directory of the ancesdir package that contains that package.json file.

However, if you want the ancestor directory of the current file or a different path, you might use ancesdir like this:

import ancesdir from "ancesdir";

console.log(`This file's root directory is ${ancesdir(__dirname)}`);

In this example, we have given ancesdir a path from which to being its search. Of course, that still only works if there is an ancestor directory that contains a package.json file. What if that's not what you want?

For the more complex scenarios, like monorepos, for example, you can use ancesdir with a marker name, like this:

import ancesdir from "ancesdir";

console.log(`The monorepo root directory is ${ancesdir(__dirname, ".my_unique_root_marker_file")}`);

ancesdir will then give you the directory you seek (or null if it cannot be found). Not only that, but repeated requests will work faster as the results are cached as the directory tree is traversed.

Conclusion

If you find yourself needing a utility like this, checkout ancesdir. I hope y'all find it useful and I would love to hear if you do. You can checkout the source on GitHub.

  1. The name is a play on the word "ancestor", while also attempting indicate that it has something to do with directories. I know, clever, right? []

Writing A Simple Slack Bot With Node slack-client

Last week, we held our first CareEvolution hackathon of 2015. The turn out was impressive and a wide variety of projects were undertaken, including 3D printed cups, Azure-based machine learning experiments, and Apple WatchKit prototypes. For my first hackathon project of the year, I decided to tinker with writing a bot for Slack. There are many ways to integrate custom functionality into Slack including an extensive API. I decided on writing a bot and working with the associated API because there was an existing NodeJS1 client wrapper, `slack-client`2. Using this client wrapper meant I could get straight to the functionality of my bot rather than getting intimate with the API and JSON payloads.

I ended up writing two bots. The first implemented the concept of `@here` that we had liked in HipChat and missed when we transitioned to Slack (they have `@channel`, but that includes offline people). The second implemented a way of querying our support server to get some basic details about our deployments without having to leave the current chat, something that I felt might be useful to our devops team. For this blog, I will concentrate on the simpler and less company-specific first bot, which I named `here-bot`.

The requirement for here-bot is simple:

When a message is sent to `@here` in a channel, notify only online members of the channel, excluding bots and the sender

In an ideal situation, this could be implemented like `@channel` and give users the ability to control how they get notified, but I could not identify an easy way to achieve that inside or outside of a bot (I raised a support request to get it added as a Slack feature). Instead, I felt there were two options:

  1. Tag users in a message back to the channel from `here-bot`
  2. Direct message the users from `here-bot` with links back to the channel

I decided on the first option as it was a little simpler.

To begin, I installed the client wrapper using `npm`:

npm install slack-client

The `slack-client` package provides a simple wrapper to the Slack API, making it easy to make a connection and get set up for handling messages. I used their sample code to guide me as I created the basic skeleton of `here-bot`.

var Slack = require('slack-client');

var token = 'MY SUPER SECRET BOT TOKEN';

var slack = new Slack(token, true, true);

slack.on('open', function () {
    var channels = Object.keys(slack.channels)
        .map(function (k) { return slack.channels[k]; })
        .filter(function (c) { return c.is_member; })
        .map(function (c) { return c.name; });

    var groups = Object.keys(slack.groups)
        .map(function (k) { return slack.groups[k]; })
        .filter(function (g) { return g.is_open && !g.is_archived; })
        .map(function (g) { return g.name; });

    console.log('Welcome to Slack. You are ' + slack.self.name + ' of ' + slack.team.name);

    if (channels.length > 0) {
        console.log('You are in: ' + channels.join(', '));
    }
    else {
        console.log('You are not in any channels.');
    }

    if (groups.length > 0) {
       console.log('As well as: ' + groups.join(', '));
    }
});

slack.login();

This code defines a connection to Slack using the token that is assigned to our bot by the bot integration setup on Slack's website. It then sets up a handler for the `open` event, where the groups and channels to which the bot belongs are output to the console. In Slack, I could see the bot reported as being online while the code executed and offline once I stopped execution. As bots go, it was not particularly impressive, but it was amazing how easy it was to get the bot online. The `slack-client` package made it easy to create a connection and iterate the bot's channels and groups, including querying whether the groups were open or archived.

For the next step, I needed to determine when my bot was messaged. It turns out that when a bot is the member of a channel (including direct message), it gets notified on each message entered in that channel. In our client code, we can get these messages using the `message` event.

slack.on('message', function(message) {
    var channel = slack.getChannelGroupOrDMByID(message.channel);
    var user = slack.getUserByID(message.user);

    if (message.type === 'message') {
        console.log(channel.name + ':' + user.name + ':' + message.text);
    }
});

Using the `slack-client`'s useful helper methods, I turned the message channel and user identifiers into channel and user objects. Then, if the message is a message (it turns out there are other types such as edits and deletions), I send the details of the message to the console.

With my bot now listening to messages, I wanted to determine if a message was written at the bot and should therefore alert the channel users. It turns out that when a message references a user, it actually embeds the user identifier in place of the displayed `@here` text. For example, a message that appears in the Slack message window as:

@here: Anyone know how to write a Slack bot?

Is sent to the `message` event as something like3:

<@U099999>: Anyone know how to write a Slack bot?

It turns out that this special code is how a link to a user or channel is embedded into a message. So, armed with this knowledge and knowing that I would want to mention users, I wrote a couple of helper methods: the first to generate a user mention embed code from a user identifier, the second to determine if a message was targeted at a specific user (i.e. that it began with a reference to that user).

var makeMention = function(userId) {
    return '<@' + userId + '>';
};

var isDirect = function(userId, messageText) {
    var userTag = makeMention(userId);
    return messageText &&
           messageText.length >= userTag.length &&
           messageText.substr(0, userTag.length) === userTag;
};

Using these helpers and the useful `slack.self` property, I could then update the `message` handler to only log messages that were sent directly to here-bot.

slack.on('message', function(message) {
    var channel = slack.getChannelGroupOrDMByID(message.channel);
    var user = slack.getUserByID(message.user);

    if (message.type === 'message' && isDirect(slack.self.id, message.text)) {
        console.log(channel.name + ':' + user.name + ':' + message.text);
    }
});

The final stage of the bot was to determine who was present in the channel and craft a message back to that channel mentioning those online users. This turned out to be a little trickier than I had anticipated. The `channel` object in `slack-client` provides an array of user identifiers for its members; `channel.members`. This array contains all users present in that channel, whether online or offline, bot or human. To determine details about each user, I would need the user object. However, the details for each Slack user are provided by the `slack.users` property. I needed to join the channel member identifiers with the Slack user details to get a collection of users for the channel. Through a little investigative debugging4, I learned that `slack.users` was not an array of user objects, but instead an object where each property name is a user identifier. At this point, I wrote a method to get the online human users for a channel.

var getOnlineHumansForChannel = function(channel) {
    if (!channel) return [];

    return (channel.members || [])
        .map(function(id) { return slack.users[id]; }
        .filter(function(u) { return !!u && !u.is_bot && u.presence === 'active'; });
};

Finally, I crafted a message and wrote that message to the channel. In this update of my `message` event handler, I have trimmed the bot's mention from the start of the message before creating an array of user mentions, excluding the user that sent the message. The last step calls `channel.send` to output a message in the channel that mentions all the online users for that channel and repeats the original message text.

slack.on('message', function(message) {
    var channel = slack.getChannelGroupOrDMByID(message.channel);
    var user = slack.getUserByID(message.user);

    if (message.type === 'message' && isDirect(slack.self.id, message.text)) {
        var trimmedMessage = message.text.substr(makeMention(slack.self.id).length).trim();
        
        var onlineUsers = getOnlineHumansForChannel(channel)
            .filter(function(u) { return u.id != user.id; })
            .map(function(u) { return makeMention(u.id); });
        
        channel.send(onlineUsers.join(', ') + '\r\n' + user.real_name + 'said: ' + trimmedMessage);
    }
});

Conclusion

Example

My `@here` bot is shown below in its entirety for those that are interested. It was incredibly easy to write thanks to the `slack-client` package, which left me with hackathon time to spare for a more complex bot. I will definitely be using `slack-client` again.

var Slack = require('slack-client');

var token = 'MY SUPER SECRET BOT TOKEN';

var slack = new Slack(token, true, true);

var makeMention = function(userId) {
    return '<@' + userId + '>';
};

var isDirect = function(userId, messageText) {
    var userTag = makeMention(userId);
    return messageText &&
           messageText.length >= userTag.length &&
           messageText.substr(0, userTag.length) === userTag;
};

var getOnlineHumansForChannel = function(channel) {
    if (!channel) return [];

    return (channel.members || [])
        .map(function(id) { return slack.users[id]; }
        .filter(function(u) { return !!u && !u.is_bot && u.presence === 'active'; });
};

slack.on('open', function () {
    var channels = Object.keys(slack.channels)
        .map(function (k) { return slack.channels[k]; })
        .filter(function (c) { return c.is_member; })
        .map(function (c) { return c.name; });

    var groups = Object.keys(slack.groups)
        .map(function (k) { return slack.groups[k]; })
        .filter(function (g) { return g.is_open && !g.is_archived; })
        .map(function (g) { return g.name; });

    console.log('Welcome to Slack. You are ' + slack.self.name + ' of ' + slack.team.name);

    if (channels.length > 0) {
        console.log('You are in: ' + channels.join(', '));
    }
    else {
        console.log('You are not in any channels.');
    }

    if (groups.length > 0) {
       console.log('As well as: ' + groups.join(', '));
    }
});

slack.on('message', function(message) {
    var channel = slack.getChannelGroupOrDMByID(message.channel);
    var user = slack.getUserByID(message.user);

    if (message.type === 'message' && isDirect(slack.self.id, message.text)) {
        var trimmedMessage = message.text.substr(makeMention(slack.self.id).length).trim();
        
        var onlineUsers = getOnlineHumansForChannel(channel)
            .filter(function(u) { return u.id != user.id; })
            .map(function(u) { return makeMention(u.id); });
        
        channel.send(onlineUsers.join(', ') + '\r\n' + user.real_name + 'said: ' + trimmedMessage);
    }
});

slack.login();

 

  1. or ioJS, if you would prefer []
  2. I find hackathons to be a bit like making a giant pile of sticks in the middle of a desert; it's an opportunity to get creative and build something where there seems to be nothing…using sticks…or in my case, a Node package and Slack []
  3. I totally made up the user identifier for this example []
  4. I used WebStorm 9 from JetBrains to debug my Node code, a surprisingly easy and pleasant experience []

CodeMash 2.0.1.5

Happy New Year, everyone! I missed my opportunity to post last week since I was on vacation at the beginning of the year and then at CodeMash immediately afterwards. This year, CareEvolution sponsored two days of NodeBot hacking during the CodeMash precompilers. I did not do a comprehensive survey, but it seemed that everyone had a fun time while learning a little something about building robots programmed using Johnny Five and NodeJS.

Besides walking around and helping people out when facing various problems with their bots, I was master of ceremonies (more commonly known as 'emcee') for the battles at the end of each day. On Thursday afternoon I presented our sponsor session on the culture we have in the CareEvolution workplace. Finally, we also had an open house on Thursday night where attendees could see some of the cool bots that had been created, ask questions, and witness a few exhibition matches.

http://instagram.com/p/xj0RWyAJ6-/

http://instagram.com/p/xhiJcTgJxl/

http://instagram.com/p/xhKBVrgJ5A/

All in all it was a busy and very successful CodeMash for everyone who helped put on all the NodeBots events. If you took part, we would love to hear what you liked or disliked about any of the NodeBots-related sessions at CodeMash this year.