Home / Testing / Parallel testing with Maven-fail-safe

Parallel testing with Maven-fail-safe

Parallel testing with Maven-fail-safe

Running a functional regression test can take up a lot of time. That is why it’s not always suitable for a buildserver as it would delay the build too much. Instead a nightly run is done to execute the functional regression test.

To speed up the execution of the functional regression test and try to fit it in the build process there are several options. One of them is implementing a Selenium Grid. This allows for parallel testing. This is fine as long as the tests run on a test envorinment but it is not recommended on buildservers due to memory issues. A better option is to configure the Maven-fail-safe plugin in such a way it allows for parallel testing

The parallel tag

In the configuration section of the maven-failsafe-plugin (in the Pom.xml) there is an option to specify how you want to test parallel. Adding the tag <parallel></parallel> will enable parallel testing.

<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<configuration>

<parallel>suites</parallel>

</configuration>

</plugin>

The parallel tag can contain multiple values. These are methods, classes, suites, classes, classesAndMethods, suitesAndMethods and suitesAndClasses. All values trigger a different way of executing the test cases.
Methods will try and run each method in parallel. This is not suitable for some cases as some of the methods used in our tests are used to setup test data for other methods. Classes however are more suitable for parallel testing.

When I first used classes (test cases) everything seemed to work fine. There were however a lot of errors that were caused because of conflicting classes running at the same time. You can control the execution order of the classes by adding the <runOrder></runOrder> tag with the values alphabetical, random, failedFirst etc. This adds some intelligence to te execution order but it doesn’t prevent classes to conflict.

The threadCount tags

In order to control the amount of parallel classes running we can add the threadCount tag in order to specify a max amount of threads running.
ThreadCountSuites determines how many parallel suites can run at the same time. There is also <threadCountClasses> if you don’t wish to use suites.


<parallel>classesAndSuites</parallel>
<threadCount>7</threadCount>
<threadCountClasses>3</threadCountClasses>
<threadCountSuites>3</threadCountSuites>

The above configuration causes the classes and suites to run in parallel. If no suites are defined a maximum of 3 classes will run simultaniously.

Controlling classes with suites

Just adding parallel and threadCount tags wouldn’t get you that far. The majority of your tests will run but you have no control over conflicting classes.
In order to precisely control the classes that can run in parallel and those that cannot I have created suites. Each suite contains a list of classes that are part of the suite.

example:

RunWith(Categories.class)
@Categories.IncludeCategory(IntegrationTests.class)
@Suite.SuiteClasses({

      BiddingProfilingWebFunctionalIT.class,
      BidOnLotPageWebFunctionalIT.class
})
public class ParallelSuiteFunctionalIT {
}

For the classes that are conflicting I have created a NotThreadSafeSuite by adding the @NotThreadSafe tag to the suite:

example:

@NotThreadSafe
@RunWith(Categories.class)
@Categories.IncludeCategory(IntegrationTests.class)
@Suite.SuiteClasses({
      DashboardPermissionRequiredWebFunctionalIT.class,
      UserPasswordRecoverWebFunctionalIT.class,
})
public class NotThreadSafeSuiteFunctionalIT {
}

in addition each class that is not allowed to run in parallel requires the @NotThreadSafe tag as well

example:

@NotThreadSafe
@org.junit.experimental.categories.Category(IntegrationTests.class)
public class AuctionOverviewWebFunctionalIT extends AbstractBvaSitePersistenceWebFunctionalIT {
...

The following configuration will be used in the pom.xml:


<parallel>suites</parallel>
<threadCount>4</threadCount>
<threadCountSuites>3</threadCountSuites>
<includes>

     <include>**/*SuiteFunctionalIT.java</include>
</includes>

Roundup

So what we have achieved now is configuring the tests in such a way that they can run in parallel in Maven. As you can see in the schema below I took you from a serial suite to parallel classes to parallel suites.
What you can also see below is that running in parallel will save time but when classes are not managed the test might fail because of conflicts.
By putting everything in suites we can run the classes in parallel where the conflicting ones will be executed at the end the parallel tests in a NonThreadSafe suite.

Now it’s up to you!

parallelvsserial

 

Interested joining BVA Auctions?

Take a look to our vacancies

About Bart Slaman

Bart Slaman is a Test Tool Consultant 10 years of experience in test automation and performance testing. He is always looking for new ways and technologies for testing. On the side he is a semi-pro photographer.

Leave a Reply

Your email address will not be published. Required fields are marked *

css.php