Behat Automated Testing

Recently I started experimenting with a tool called behat, which was configured for us and recommended by Chris Sheppard, Tribal's previous Head of HTML Development, and used heavily by our Head of LAMP on some PHP projects. It is a behavior driven development tool that allows you to create automated functional testing scripts from User stories.

I was very skeptical about how well it would actually function. Automated testing tools often promise big things, but fail to deliver when they are deployed on a large scale corporate site, because they can't cope with bespoke implementations or with rapid content changes.

Behat works on essentially two levels. "Features" are user stories. Scenarios, which are specific tests, and Scenario Outlines, which are tests that use a defined set of examples to be run across multiple pages. For example we are able to write a simple user story like this

Scenario Outline: Car landing pages have content
Given I am on "/new//home"
Then I should see the vw-web page layout
And I should see "title" in the ".hero h2" element

Examples:
| link | model-web-name | title |
| up! | up-nf | up! |
| Polo | polo-v | Polo |
| Golf | golf-vii | Golf |

Which will then iterate through each line in the examples as a new test.

The second level is the PHP context files that interpret the Features. One of the many nice things about the Behat approach is that is allows the user stories to drive the PHP rather than the other way round. The users create the user stories and those lines which the context files cannot interpret are presented with a 'best guess' on what should be implemented. This means the full suite of tests can be drafted and then a developer can be brought on board to implement them.

A second element to this approach that works very well is that as you realize standard elements to the request, then can be added into a single request. For example, we have a simple line of "I should see the vw-web page layout" which looks for particular elements on that page unique to the vw site to confirm the basics chrome of the page is rendering. We noticed we were adding "And I should not see "404"" regularly to the user story and so we moved this requirement into the list of elements tested when a user enters "I should see the vw-web page layout".

One thing Project Managers need to be aware of when using this tool is that it can become to over "tech" the user stories, saying things like "I should see title in the h1 element" This instead should be "I should see title in the title" and the php should define that title means a h1 tag. This ensures that the user stories remain accessible to the non-technical team, as they should be. It also helps to encourage a standard style. If you are using the term "title" and that always means h1, then you avoid the trap of letting the code define the implementation. I.e. the developer makes the heading a h2 tag and the tests get updated.

Our setup leverages Saucelabs to allow us to run the tests in various browsers and means we are able to test pages with JavaScript in them. We also have Guzzel for tests without JavaScript, as a local headless browser is much quicker to run through the stories. Its then run either manually or automatically through our CI server; Jenkins.

Two items we need to move onto using are the Jira integration that is offered by Behat, as much of our planning and user story creation is done as jira tickets with the type of "User Story" and so, as part of application development, we are able to define the tests that need to be passed and agree them with the client before development even starts. They become our definition of done.

Secondly, we need to create a mechanism for importing data, like offers, model details and other items that are used heavily across much of the site, so we do not have to have separate entries in a range of files with the same information.

All told, I've been wildly impressed with Behat, to the point of annoying the people around me talking about it.

Try it, its really, really powerful and brings together the project team and helps us Define Done.

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.