Running Cucumber JVM tests in parallel

View the updated version of this blog post here!

Addressing slow test runs when using Cucumber JVM with web automation tools

UPDATE March 2016: While the content of this post still accurately describes an approach to running cucumber scenarios in parallel, the associated sample code is by now somewhat out of date. This topic has been revisited in a new blog post which incorporates new tools and up to date versions. The sample code for the new post is part of the quick-start test framework and as such is being actively maintained.

Getting Started

The full source code for this example can be found on Github.

You will need to have Maven and Firefox installed on your machine in order to run this example.

Once you have retrieved the source, you can run by navigating into the Cucumber-JVM-Parallel directory and issuing the command:

mvn clean install

This will run the example project and start two browser windows concurrently. Each browser window corresponds to a Cucumber feature file in the example. Once the run is complete, a report will have been generated at /target/cucumber-report/index.html.

The rest of this post will go into further detail on the structure of the example project, though it is assumed that you have some prior experience of Cucumber. If not, the Cucumber-JVM readme is a great place to start.

Feature files

The first thing you need is your feature files to describe the behaviour you expect. In this example, we have two separate features, though you can also run scenarios within a single feature in parallel.

The way we can do is is by using Cucumber tags, which can be applied either to all scenarios in a feature or to individual scenarios.

Above, you can see that we have two feature files. These reside in the ‘src/test/resources’ folder. Each feature file is tagged (@autocorrect and @search), and contains a single scenario.

Glue code

Now that we have our scenarios, we need to add some glue code to tie each step into our underlying test framework

These are referred to as step definitions, and can be found in ‘src/java/cucumber.jvm.parallel/cucumber/stepdefs’.

In the above snippet, you can see that we use an instance of ShareDriver to communicate directly with a browser window. This is based on one of the cucumber examples for sharing a single browser session between all tests (using dependency injection) to remove the need for starting up a browser instance per test and thus speed up execution. In our case, this results in one browser session per thread. The ShareDriver class can be found in ‘/src/test/java/cucumber.jvm.parallel/cucumber’.

Page Objects

The snippet also shows that we use an instance of ‘SearchPageObject’, which is simply a class which represents the Google search page,  found in ‘/src/test/java/cucumber.jvm.parallel/pageobjects’.

This is not required, but it’s good practice to use the page object pattern for ease of maintainability on a larger project.

Above, you can see that the page object contains identifiers for elements on the page as well as methods specific to that page.

Runners

The next stage in the process is adding the test runners. We are using JUnit as opposed to the CLI, and this is where we need to start structuring things specifically to handle parallel running of tests.

In the above snippet from ‘SearchAT.class’ you can see that we are specifying the location of the feature files. We are also specifying a tag (@search) which relates to one of our cucumber feature file tags and a html report destination for test results.

What this says is “run all tests tagged as @search and write results to /search folder”.

We then have another class, ‘AutoCorrectAT’ which does the same for all tests tagged ‘@autocorrect’. Both of these classes can be found under ‘/src/test/java/cucumber.jvm.parallel/cucumber’.

Adding another thread is simply a case of adding a new runner class with a different tag.

Parallel Test Runs

Up to this point, the instructions are identical to creating a relatively simple non-parallel set of Cucumber-JVM tests using WebDriver to interact with a website.

We now need to go to the Maven POM file to see how we are making the tests run in parallel.

In the above snippet, you can see that the maven-surefire-plugin is used to run our acceptance tests – any classes that end in *AT will be run as a JUnit test class. Thanks to JUnit, making the tests run in parallel is now a simple case of setting the forkCount configuration option. In the example project, this is set to 5, meaning that we can run up to 5 threads (ie, 5 runner classes) at a time.

Reporting

Running the test suite at this point would result in our two feature files being run in parallel, with their results being written out to separate reporting directories. For ease of reporting, our final step is to merge these reports into a single file. To do this, we have some code in ‘/src/main/java/cucumber.jvm.parallel/ReportMerger’. This is executed by the exec-maven-plugin in the Maven POM.

The report merger code consists of the following steps:

• For a given report directory, go through each sub directory:

  • Rename all embedded images to have a unique name (without this there would be clashes between different reports being merged).

  • Update the report.js file to reflect the new embedded image file names.

  • Copy renamed images to root report directory.

  • Merge report.js into file in root report directory.

The result of this is a single index.html file in the root report directory which consists of all subreports. The subreports are left in place, but these could easily be deleted at this point if you are archiving test results on your CI server and wish to save space.

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.