Getting Started with Selenium Load Testing

Selenium Flood Tests

Selenium helps you test web applications by simulating how a real user would interact with your application using real browsers. The WebDriver API has language bindings and implementations to help drive different browsers.

Combine Selenium with WebDriver on Flood and you get distributed load testing across the globe.

Chrome and Firefox

On Flood you can launch browsers using ChromeDriver or FirefoxDriver that run in isolated Docker containers.

Since these containers run full browsers, CPU overheads are naturally higher so we limit the amount of browsers to 5 per grid node.

The benefit of running full browser automation for load testing comes from driving the browser itself. Being able to run existing browser based automation suites as a control sample in conjunction with more traditional load testing tools and techniques is valuable for the increased fidelity of the simulation. However there's nothing stopping you from running larger grids with multiple nodes to increase the level of concurrency if you want to run Selenium only load tests.

Java Bindings

We currently support native Java bindings for WebDriver so you can re-use existing tests already written in Java. Depending on demand, we may support additional bindings / test runners in other languages in future. At present we'll stick with Plain Old Java Objects. Let us know...

Example Test

For your tests to work on Flood you'll need to import our FloodSump. This is a custom plugin that lets us pull together reporting information for your flood tests. The critical functions your script needs to call are .started() and .finished() for test boundaries. Inside that you can make calls to .passed_transaction(WebDriver driver, String label, Integer responseCode) or .failed_transaction(WebDriver driver, String label, Integer responseCode).

When your test is running we'll pass in the relevant WEBDRIVER_HOST and WEBDRIVER_PORT environment variables for your test to connect to.

Have a look at the following example test plan.

js
import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;

import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriverException;

import org.openqa.selenium.remote.Augmenter;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

import org.openqa.selenium.support.ui.Select;

import io.flood.selenium.FloodSump;

public class Loadtest  {
 public static void main(String[] args) throws Exception {
   int iterations = 0;

   // Create a new instance of the html unit driver
   // Notice that the remainder of the code relies on the interface,
   // not the implementation.
   WebDriver driver = new RemoteWebDriver(new URL("http://" + System.getenv("WEBDRIVER_HOST") + ":" + System.getenv("WEBDRIVER_PORT") + "/wd/hub"), DesiredCapabilities.chrome());
   JavascriptExecutor js = (JavascriptExecutor)driver;

   // Create a new instance of the Flood IO agent
   FloodSump flood = new FloodSump();

   // Inform Flood IO the test has started
   flood.started();

   // It's up to you to control test duration / iterations programatically.
   while( iterations < 1000 ) {
     try {
       // And now use this to visit the target site
       driver.get("https://loadtest.flood.io/usertiming");

       // Log a passed transaction in Flood IO
       flood.passed_transaction(driver);

       // Log a custom mark from the User Timing API
       flood.get_mark(driver, "mark_headers_loaded");

       // Log a custom measure from the User Timing API
       flood.get_measure(driver, "measure_page_load");

       iterations++;

       // Good idea to introduce some form of pacing / think time into your scripts
       Thread.sleep(3000);
     } catch(InterruptedException e) {
       Thread.currentThread().interrupt();
       String[] lines = e.getMessage().split("\\r?\\n");
       System.err.println("Browser terminated early: " + lines[0]);
     } catch (WebDriverException e) {
       String[] lines = e.getMessage().split("\\r?\\n");
       System.err.println(lines[0]);
     }
   }

   driver.quit();

   // Inform Flood IO the test has finished
   flood.finished();
 }
}

How it works

We use Docker containers that run standalone Selenium Node images provided by the Selenium project for Chrome and Firefox. Inside those our Java plugin makes calls to the same internal reporting API using Elastic that all of our load test runners use. The plugin itself is using the JavascriptExecutor within WebDriver to parse the Navigation Timing API, something along the lines of:

js
JavascriptExecutor js = (JavascriptExecutor)driver;

long loadEventEnd = (Long)js.executeScript("return window.performance.timing.loadEventEnd;");
long navigationStart = (Long)js.executeScript("return window.performance.timing.navigationStart;");
long responseStart = (Long)js.executeScript("return window.performance.timing.responseStart;");
long connectStart = (Long)js.executeScript("return window.performance.timing.connectStart;");
...

Beyond that, results are formatted in the same fashion as all of our load test reports. Importantly we're not relying on any proxies in between your browser and the target site so as to minimize impact on your test results as much as possible.  More information on getting started can be found in our getting started guide for selenium.

We'd love your feedback on Selenium Load Testing with Flood.  We're always interested in your feedback and requests for different ways to generate load.

‍