Dream of Browser Test Automation?


Browser testing has been a pain for a decade now. The landscape is showing a lot of promise nowadays, and I wanted to share the approach to client-side testing I’ve taken with dragula. In this article, you’ll see what’s so wonderful about Tape, why TAP is the future of browser testing, and how you can leverage it to the fullest in both development flows and continuous integration environments such as Travis CI.

By now you must’ve read how I use tape for all the things, or how others do as well. Am I the only one here amused that we wrote almost identical articles about Tape, and published them on the same day? We’ve already covered why Tape is better than Mocha et al. It’s modular, it’s sensical, and it produces TAP output.

When it comes to server-side code, consider a module like this:

function greet (name) {
  return 'hi ' + name;
module.exports = greet;

A test written in tape might look like:

var test = require('tape');
var greet = require('../greet');

test('greeting produces a salutation', function (t) {
  t.equal(greet('stranger'), 'hi stranger', 'greeting is proper and amicable');

Running the above test might involve adding a test script to your scripts in the package.json manifest, like below.

  "scripts": {
    "test": "node test/*.js"

I cheated, and that piece of code will run any test files inside the test directory. To run that, simply use the npm run alias for test.

npm test

How do we deal with the DOM, continuous testing during development hours, and continuous integration on remote server? For the answers, we might want to go back to the seemingly innocuous TAP output from tape.

The beauty of TAP

TAP being a precisely defined format means that any programs that can process or output TAP can be connected through UNIX pipes. Like most good things in programming, this is deceitfully simple. Consider, for example, the case of reporters. There are over a dozen TAP reporters listed on the tape. This is a familiar concept in test tooling. The tool generates output that can then be formatted, in turn, by a reporter that highlights the types of information we care about (failing tests, diffs, nyan cats, etc). TAP can also be used for transformations, such as the hacks code coverage tools like coverify do to figure out how good your code coverage is.

Then there’s continuous testing flows and continuous integration. Let’s start with the former.

Continuous Testing with hihat

hihat is a shiny new package created only two months ago, but it’s potential and usability are both huge. Let me just demonstrate using a screenshot.

A gif depicting hihat in action
A gif depicting hihat in action

You may have noticed that in the demo they use tap-dev-tool. This is a reporter that prettifies TAP for our particular use case – rendering the tests on a DevTools instance. The way hihat works is really cool. It uses Electron behind the curtains, the runtime used under GitHub’s Atom editor. In case it’s not obvious from the demo, hihat reloads the DevTools whenever you make a change to your code.

If you’re anything like me, your first impression when you saw that screenshot was “that’s too good to be true”, and your second impression probably was something along the lines of “there’s no way that’s easy to set up”.

Turns out, it’s freaking easy to set up. First, install things.

npm i -D hihat tap-dev-tool

In case you were wondering, i is one of plenty of shortcuts for install, and -D aliases --save-dev.

Then, make a slight change to your package.json.

  "scripts": {
    "test": "hihat test/*.js -p tap-dev-tool"

Now you can just run the tests that you used to run simply in Node, within a DevTools instance (effectively, in Chromium), and it will watch for changes! Super amazeballs.

Continuous Integration with testron

A very similar and complementary tool to hihat is testron. Testron takes the output from your browserify build and runs it through electron as well. Testron is designed to run on the command-line, so you won’t get any fancy DevTools. But, Testron does make it dead simple to run your browser tests on Chromium.

Just like hihat, testron is dead simple to use.

browserify test/* | testron

That’s it!

Testron running our client-side tests on Travis
Testron running our client-side tests on Travis

Bonus: errorify

During development, you can save precious think-shark-tank seconds by using errorify. It’s meant to be used together with watchify, and it’ll tee compile errors into the terminal and a bundle whose sole purpose is to tell you that the bundle failed to compile from within the HTML.

This way, you can save some time figuring out “oh, it failed to compile, probably missed a semicolon”, or similar situations. Stop tabbing between your terminal and your browser today!

Liked the article? Subscribe below to get an email when new articles come out! Also, follow @ponyfoo on Twitter and @ponyfoo on Facebook.
One-click unsubscribe, anytime. Learn more.