Open Credo

April 5, 2016 | Software Consultancy

Test Automation Concepts – Test Data and Aliases

This post is part of a series which introduce key concepts in successful test automation. Each post contains sample code using the test-automation-quickstart project, a sample Java test automation framework available from Github.

WRITTEN BY

Tristan Mccarthy

Tristan Mccarthy

Test Automation Concepts – Test Data and Aliases

For a full list of concepts introduced so far, see the parent post “Introducing the test automation quick start project”.

Test data and aliases

The first step of successful test automation is the ability to interact with the system under test to change state and make assertions. Once this is in place there is a new problem to deal with, which is how to handle test data. For successful automation of any sort, it’s important to ensure that tests are isolated: that they do not share any state or affect one another. This is particularly important when writing tests that can run in parallel.

The most common mistake is writing tests in such a way that the order in which they run is important. If you have a “Test B” which expects that the data be in the state left by the successful execution of “Test A” then any failure in the chain causes all other tests to fail. Using a small set of static test data also means that you need to maintain a separate environment just for automated tests which can be rolled back to a known state for every run, and anyone else interacting with the system at the same time could affect the outcome of the tests.

Overcoming this brittleness can be either easy or near impossible depending on the nature of the system under test, but my preferred method of handling this (and other related problems of static data sets) is to have every test generate its own state by randomly generating test data. This often involves some initial investment in test infrastructure to ensure that there are mechanisms for easily creating test data, but it pays dividends over time as the tests become far more stable and maintainable. As an aside, injecting data directly into databases should be a last resort as it adds a new element of maintenance to keep in line with changing schemas, as well as bypassing any validations imposed by services communicating with that database. The cleanest possible approach is to use the same APIs used by whatever application you are testing, but this is dependent on the architecture of the system you are interacting with.

Dynamically creating data does come with a time cost, especially if the performance of the application makes creation a lengthy process. However, with tests that are subsequently able to run in parallel rather than consecutively the net result is a far quicker and more stable set of tests. For more on running tests in parallel see our recent blog post covering this topic.

If you are engaging the business in your testing process with Behaviour Driven Development (BDD), using dynamic data brings a new challenge in making sure that the steps remain clear and readable. For simple scenarios the problem is rarely apparent but as we begin to test more complex business problems the lack of a concrete entity to refer to can begin to affect clarity. This is where we can introduce the concept of aliases to allow us to refer to business entities in a more natural way.

Getting Started

In the rest of this post, we’ll be working through an example to demonstrate the use of aliases with an example test framework.

To begin with, download the test-automation-quickstart project from the OpenCredo GitHub account. Once done, follow the instructions on the ReadMe and ensure that you have the required dependencies installed (Maven, FireFox). For those who want to dive straight in at this point, you can run the code with the following console command from the project root directory:

mvn clean install

This will result in the full set of tests in the example project running successfully.

Defining the test

When testing with BDD the start point is always the human readable scenarios themselves. To begin with, we can demonstrate the evolution of a scenario from a simple test including static data through to something more dynamic data friendly using aliases. We’ll start with an example which is of a style similar to that common among testers new to BDD. The system under test, in this case, is a simple messaging application.

# scenario which uses static data
Scenario: Send a message between two users
  Given I am logged in as "Bob" with password "password"
  When I send a message to "Dave"
  Then "Dave" should have received a new message from "Bob"

The above test makes the assumption that certain users exist in the system and makes explicit reference to the data. It is clear and easy to understand at this point, but suffers from all of the problems of static data discussed in the previous section. Our first step should be to shift away from the assumption of data and towards a more dynamic-friendly approach.

# scenario which uses nonspecific data
Scenario: Send a message between two users
  Given I have registered as a new user
  And another registered user
  When I log in
  And I message the other user
  Then the other user should have received a message from me

The test no longer makes reference to specific data. The implementation code of the test can go away and generate the test data and store it internally for use in the test steps. However, the transition has definitely lost some of the clarity of the original test, and this is on a relatively simple piece of business logic – though on an example chosen to highlight the problem that can be encountered. We can now introduce the alias approach.

# scenario which uses aliased data
Scenario: Send a message between two users
  Given the following users:
    | Bob   |
    | Dave  |
  When I log in as "Bob"
  And I send a message to "Dave"
  Then "Dave" should have received a message from "Bob"

Our test now looks very similar to how it started, with the major difference that the test now explicitly states its precondition that certain users exist in the system. We now have a test that remains clear for a nontechnical reader but supports the use of dynamic test data in the implementation layer.

Creating and storing the data

We can now dive into our code base and look at how we can generate and store our aliased test data. Our scenario has to first be included in a Cucumber feature file so that it will run within the test framework.

./test-automation-quickstart/api-acceptance-tests/src/test
    /resources/cucumber/AliasExample.feature
@ui-demi-alias
Feature: Demonstrate use of aliases to keep tests readable with complex data
  
  Scenario: Send a message between two users
    Given the following users:
      | Bob   |
      | Dave  |
    When I log in as "Bob"
    And I send a message to "Dave"
    Then "Dave" should have received a message from "Bob"

The first step in the scenario creates the data in the system under test and stores it so that it can be referenced later on in the tests. We are treating this test as an API test for the purposes of implementation. First is the creation of the step definition, the link between the human readable BDD step and underlying code.

./test-automation-quickstart/api-acceptance-tests/src/test
    /step/defs/MessagingStepDefs.class
@Given("^the following users:$")
 public void theFollowingUsers(List userAliases) throws Throwable {
   for(String userAlias : userAliases) {
     createUser(userAlias);
   }
 }

private void createUser(String userAlias) {
  User newUser = new User();
  messagingApi.registerUser(newUser, testWorld.USER_PASSWORD);
  testWorld.addUser(userAlias, newUser);
}

Our step definition takes the list of user aliases and for each one calls our createUser helper method. The helper method performs three distinct operations. Firstly it generates a new User object, a simple object which consists of basic user information – first name, last name, and username. The information is randomly generated on instantiation.

We then interact with our system under test to register a new user. The MessagingApi class is an object which represents the API of the system and provides us with generic interaction methods. In this particular case, there is no real system so the class actually contains a static list of users which it adds to. The call to add a new user passes through a static variable for the password, which is stored in TestWorld. This is a class which allows us to store data that we wish to share between steps – it is possible to save this data in the StepDef class itself, but as the test framework becomes more complex you will find that often you will need to share test state across multiple StepDef classes.

The final step is to add the newly created user to our testWorld, using the alias as the identifier. This will allow us to retrieve the details of the generated user later using the alias. The important thing to understand here is that in a real test we would have both the real data added into the system and our representation of the data which is stored within the test framework. As we interact with the system and change the state of the data, we need to ensure that we keep our representation up to date.

Referencing the data

With our test data in place, we can now perform actions within the system and make our assertions to ensure that the behaviour matches our expectations.

The next part of the test is to log in and send a message between our two users.

./test-automation-quickstart/api-acceptance-tests/src/test
    /step/defs/MessagingStepDefs.class
@When("^I log in as \"([^\"]*)\"$")
 public void iLogInAs(String userAlias) throws Throwable {
   logIn(userAlias);
 }

 private void logIn(String userAlias) {
   User user = testWorld.getUserByAlias(userAlias);
   user.authToken = messagingApi.authenticate(user.userName, testWorld.USER_PASSWORD);
   testWorld.currentUserAlias = userAlias;
   testWorld.addUser(userAlias, user);
 }

For the log in step we pass the user alias through to another helper method. This method uses the alias to retrieve the dynamically generated user from our TestWorld, then interacts with the system through our API object to authenticate. The example makes an assumption that the result of this interaction is an authorisation token which can be used to perform subsequent interactions which require authorisation.

The authorisation token is then added to the user before being saved back into our test world so that we can use it later. We also keep a reference to the currently logged in user so that we can retrieve it later without the BDD scenario needing to pass that information through.

./test-automation-quickstart/api-acceptance-tests/src/test
    /step/defs/MessagingStepDefs.class
@When("^I send a message to \"([^\"]*)\"$")
public void iSendAMessageTo(String userAlias) throws Throwable {
  User recipient = testWorld.getUserByAlias(userAlias);
  User sender = testWorld.getCurrentUser();
  String message = RandomUtils.randomAlphaString(20);
  messagingApi.sendMessage(sender.userName, sender.authToken, recipient.userName, message);
}

We now retrieve two User objects from TestWorld – one from the alias given as the message recipient and the other from the currently logged in user. We then randomly generate the message body and use the messaging API again to send the message along with all other details required by the system. We should now have created a message between the two users and are able to assert that the behaviour matches our expectations.

./test-automation-quickstart/api-acceptance-tests/src/test
    /step/defs/MessagingStepDefs.class
@Then("^\"([^\"]*)\" should have received a message from \"([^\"]*)\"$")
public void shouldHaveReceivedAMessageFrom(String recipientUserAlias, String senderUserAlias) throws Throwable {
  User recipient = testWorld.getUserByAlias(recipientUserAlias);
  User sender = testWorld.getUserByAlias(senderUserAlias);
  assertThat(messagingApi.getMessages())
   .filteredOn("recipientUserName", recipient.userName)
   .extracting("senderUserName")
   .contains(sender.userName);
}

Our final step retrieves the two User objects again before retrieving messages for the expected recipient and using assertJ to check that the message list contains our expected message from the sender. As a side note, in a real world situation we would also want to assert that the message body is as we expect. The message could be stored as an aliased object in the same way as the users.

Conclusion

We now have a test which is entirely self-sufficient. It can be run independently or as part of a set. We could run the same tests 100 times in parallel and each one would pass. The test can be run while another user is manually interacting the system and it will have no effect on the results. You can choose to clean the data up after the test has completed using the @After Cucumber hook, or leave it safe in the knowledge that it won’t have any impact on subsequent test runs (assuming there are no performance implications of continuously creating new data in the system).

While some aspects might seem overly complex for the simple scenario we are addressing, the approach used (especially storing our aliased objects in TestWorld) result in a structure which remains maintainable as the test suite grows.

 

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.

RETURN TO BLOG

SHARE

Twitter LinkedIn Facebook Email

SIMILAR POSTS

Blog