Sử dụng CasperJS, Drush and Jenkins để test Drupal

A few weeks ago, spurred on by some frustrating interactions with Behat and Javascript, I started looking into alternative frameworks for running behavioral tests against Drupal. A little bit of searching led me to this helpful post Behavorial Test For Custom Entity Using CasperJS, which outlines the basics of writing some simple CasperJS tests for a site.

I had never heard of CasperJS before, and if you haven't either you might want to go check out the CasperJS site then come back here. In short, CasperJS has a couple things going for it:

• It's very fast. I haven't run any benchmarking tests but it's clearly much faster than Behat + Selenium
• It's built on top of the excellent headless WebKit, PhantomJS
• Writing tests is easy; it's all in Javascript. Even if you are a JS novice, the documentation for the project is superb and can get you up and running quickly

On the other hand, Behat is a more mature framework and has the advantage of being written in PHP, plus, there is the outstanding Drupal Extension which provides Drush integration and simplifies content and user creation / deletion. For beginners you also have a library of tests to reference in the Drupal.org BDD tests project a.k.a. "Doobie".

With CasperJS, Drush is not available to us. But for our tests to really be useful, we'll need a way to access Drush during the test process, and we'll need a way to automatically create our own users and content and do other setup/teardown tasks. Clearly we don't want to write dozens of steps for submitting user/content creation forms; we want to use Drush. This could be done by placing a list of Drush commands in setup.shand teardown.shbash scripts, but this still doesn't solve the problem of accessing Drush in the middle of a test. So how can this be done?

Run Drush in the browser

Drush has a (probably not that well-known) command for running a web server on your machine: drush runserver. It turns out, extending the runservercommand to expose Drush as a web service is really easy; instead of serving up a Drupal site with runserver, we will provide interaction with Drush.

You can try it out for yourself. Grab this composer.json file and place it somewhere on your machine. Then switch to that directory and run composer install; this will download a customized Drush install into vendor(you can see and contribute to the pull request to Drush 7 here). Now run vendor/bin/drush runserver --drush-server; then visit http://localhost:8888/core-status --jsonand you'll see the output of Drush's core-status command in your browser, in JSON format, it should look something like:

{
    "php-bin": "/usr/bin/php",
    "php-os": "Linux",
    "php-conf": [
        "/etc/php5/cli/php.ini",
        "/home/kosta/.drush/drush.ini"
    ],
    "drush-version": "7.0-dev",
    "drush-conf": [
        "/home/kosta/.drush/drushrc.php"
    ]
}

With the drush runserver --drush-servercommand, any time you pass the

--jsonoption to Drush in the browser, you'll get JSON content back that can easily be parsed using Javascript. Commands that don't return output (like drush cc all) will return a JSON encoded success / error message depending on their results. For example, visiting http://localhost:8888/made-up-commandwill return a 500 error code along with:

{
    "DRUSH_COMMAND_NOT_FOUND": [
        "The drush command 'made-up-command' could not be found. Run `drush cache-clear drush` to clear the commandfile cache if you have installed new extensions."
    ],
    "DRUSH_NO_DRUPAL_ROOT": [
        "A Drupal installation directory could not be found"
    ]
}

This is great because in CasperJS, you can, for example, test creating some content then call drush watchdog-show --format=jsonto check the logs and see if an email was correctly generated and format - much like you can with Behat.

Even better, this means that now with we can include a setup.jsfile. CasperJS lets you specify --preand --postoptions to include tests that run before and after your main test suite.

So we can do casperjs test --pre=setup.js --post=teardown.js /path/to/tests/dir. Then in our setup.jsfile, we can have code like:

var uri = casper.cli.get('uri');
var link = 'http://' + uri;
var alias = casper.cli.get('alias');
var drush = 'http://localhost:8888/' + alias + ' ';
var repo_dir = casper.cli.get('repo');

casper.test.begin('Setup methods for test suite', function suite(test) {

  // Begin setUp().
  casper.start(drush + 'core-status --format=json', function() {
    this.test.assertTextExists('Successful', 'Bootstrapped to Drush Runserver');

    // Online Help: Cancel/create test accounts.
    this.open(drush + 'user-cancel testOnlineHelpSchoolAdmin --delete-content');
    this.open(drush + 'user-cancel testOnlineHelpAdmin --delete-content');
    this.open(drush + 'user-create testOnlineHelpSchoolAdmin --password=test');
    this.open(drush + 'user-add-role "School Administrator" --name=testOnlineHelpSchoolAdmin');
    this.open(drush + 'user-create testOnlineHelpAdmin --password=test');
    this.open(drush + 'user-add-role "Administrator" --name=testOnlineHelpAdmin');

    // etc etc.

    this.test.comment('Done with setUp methods.');
  });
  // end setUp().

  casper.run(function() {
    this.test.done();
  });

});

Tip: give unique names for your test users based on the test that's being run (i.e. "testOnlineHelpAdmin" instead of "testAdmin"). Why? Because with Grunt and the grunt-casperjs plugin, you can run CasperJS tests asynchronously. Having different usernames per test will allow you to avoid collisions with CasperJS trying to simultaneously use the same user account for multiple test scenarios.

Finally, here's an example test - a few simple checks that only users with the Administrator role can access our Drupal site's "Online Help" is below:

// Test Access to the online help section.
// The online help handbook should only be accessible to administrators.

casper.test.begin('Restrict access to online help to Administrators only', function suite(test) {

  // Check that online help is not accessible to anonymous user.
  casper.start();
  casper.setHttpAuth('USERNAME', 'PASSWORD');

  casper.thenOpen(link + '/handbook').waitForSelector('.error',
    function then() {
      this.test.assertTextExists('Access denied', 'Anonymous user denied access to restricted node.');
      this.test.assertExists('#user-login', 'Login form present on Access Denied page');
    },
    function timeout() {
      this.capture('/tmp/help-access-denied.png');
      this.test.assert(false, 'Loaded user login form and displayed access denied');
    }
  );

  // Logout
  casper.thenOpen(link + '/user/logout', function() {
  });

  // Check that online help is not accessible to School Administrator.
  casper.thenOpen(link + '/user').waitForSelector('#user-login',
    function then() {
      this.fill('form#user-login', {
          'name': 'testOnlineHelpSchoolAdmin',
          'pass': 'test'
      }, true);
    },
    function timeout() {
      this.test.assert(false, 'Failed to load login page.');
    }
  );
  casper.thenOpen(link + '/handbook').waitForText('Access denied',
    function then() {
      this.test.assertTextExists('Access denied', 'School Administrator user denied access to restricted node.');
    },
    function timeout() {
      this.capture('/tmp/handbook-access-denied.png');
      this.test.assert(false, 'Loaded login form present on access denied.');
    }
  );

  // Logout
  casper.thenOpen(link + '/user/logout', function() {
  });

  // Check that online help is accessible to Administrator.
  casper.thenOpen(link + '/user').waitForSelector('#user-login',
    function then() {
      this.test.assertExists('form#user-login', 'Loaded login page.');
    },
    function timeout() {
      this.test.assert(false, 'Loaded login form.');
    }
  );

  casper.then(function() {
    this.fill('form#user-login', {
      'name': 'testOnlineHelpAdmin',
      'pass': 'test'
    }, true);
  });

  // Access handbok as Admin user.
  casper.thenOpen(link + '/handbook').waitForText('Online Help',
    function then() {
      this.test.assertTextExists('Online Help', 'Admin can access Online Help');
      this.test.assertDoesntExist('#user-login', 'Login form not present on handbook page');
    },
    function timeout() {
      this.capture('/tmp/online-help-not-loaded.png');
      this.test.assert(false, 'Loaded Online Help for administrator.');
    }
  );

  // Logout.
  casper.thenOpen(link + '/user/logout', function() {
  });

  casper.run(function() {
    this.test.done();
  });
});

Running the test with casperjs test tests/casperjs/online-help.js
--pre=tests/casperjs/helpers/setup.js
--post=tests/casperjs/helpers/teardown.js --uri=mysite.localhostthen results in this output:

Test file: tests/casperjs/tests/online-help.js
#CDCD00;"># Restrict access to online help to Administrators only
PASS Anonymous user denied access to restricted node.
PASS Login form present on Access Denied page
PASS School Administrator user denied access to restricted node.
PASS Loaded login page.
PASS Admin can access Online Help
PASS Login form not present on handbook page

Putting it all together with Jenkins CI

Behavior driven development is a great software development process. You can spec out a feature, write the tests for it, then write the code to implement the feature. Or you can write tests against an existing set of features to catch regressions.

One thing I've learned, though, is that unless you have a policy for running tests, and enforce that policy, the tests won't get run; they'll eventually become out-dated, and then your tests are no longer useful.

This is where Jenkins comes in. While it's out of the scope of this post to write out how to set up Jenkins, this guide, Drupal Continuous Integration with Jenkins is really helpful and you can be up and running within a few hours. With a little bit of work, I configured a Jenkins instance so that on each commit that's pushed to our git repository, Jenkins will (1) import our @proddatabase and files to our @testinstance on Acquia Dev Cloud (check out this gist for an example python script that will poll Acquia Cloud to see when the DB import is done, so you don't start your tests before the DB import task finishes!), (2) check out the git branch that was committed to in the @testenvironment, (3) run some setup tasks using Drush, (4) execute the CasperJS tests, and (5) run some teardown tasks. If any part of this fails, e-mails are sent out to the project manager and the committer with details about the failed build, so it can be fixed.

Questions? Ideas on how to do things differently? Please let us know in the comments.