HP LoadRunner Alternatives: Comparing Flood vs. StormRunner

Reasons you should consider Flood over StormRunner (and LoadRunner)

Here at Flood, we are asked every day for replacement options for HP LoadRunner (now owned by MicroFocus).  While LoadRunner has been the leading load testing tool in the market for many years, it has lost favor with many teams over the last few years for a few key reasons:

  • Cost: HP LoadRunner typically costs in the mid six figures for an average size deployment.  If you are looking at a large deployment, seven figures is not out of question.
  • Perpetual Licensing: typical LoadRunner licenses are still perpetual and difficult to scale up and down
  • Decreased Need for Non HTTP Protocols: with the migration of many packaged applications to the cloud, companies have less need for the wealth of protocols that LoadRunner can support

Replacing HP LoadRunner with StormRunner

HP, being wise to the changes in the market, decided to introduce StormRunner as their own LoadRunner alternative.  The main aims of StormRunner were to provide a solution that had:

  • Native cloud integration
  • Support for popular open source tools like JMeter
  • More flexible, consumption based licensing

StormRunner quickly became a popular replacement solution to customers that had become disillusioned with LoadRunner.  

Comparing Flood vs. StormRunner

Features

StormRunner and Flood both rely on a similar set of features in order to deliver a seamless testing experience in the cloud.

Shared Features Between StormRunner and Flood

Both tools offer a robust set of cloud functionalities:

  • Browser based load testing (TruClient for StormRunner, Element for Flood)
  • JMeter, Gatling, Selenium Integration
  • Analysis Reporting
  • APM Integrations to tools like AppDynamics, New Relic, and Dynatrace
  • Ability to execute tests in a shared cloud
  • Ability to execute tests in a private cloud account with AWS or Azure
  • REST API for integration to DevOps Pipeline

StormRunner Specific Features

StormRunner offers some unique features not found in Flood, including:

  • Support for a wider variety of protocols (non-web like SAP GUI)
  • Ability to use LoadRunner tests
  • Sitescope, an dedicated monitoring solution to use in place of AppDynamics, New Relic, and Dynatrace

Flood Specific Features

Flood offers some unique features not found in StormRunner, including:

Pricing

While the pricing models for StormRunner and Flood are similarly consumption based, they do vary in that Flood is truly consumption based (on the # of servers used to test) vs. StormRunner is based on the number of Virtual User Hours consumed (which is an approximate representation of the # of servers used).

Protocol Level Comparison: JMeter

Looking at the cost of an identical test, which might be a one time test with 20,000 users in JMeter for 1 hour, the costs would be as follows:

  • Flood: 20,000 users for 1 hour would require 20 nodes (load injectors) for 1 hour, which comes to a total cost of $149
  • StormRunner: 20,000 users for 1 hour would require 20,000 virtual user hours (Web), which comes to a total cost of $4,500

Verdict: Flood is 30x less expensive than StormRunner in this Protocol Level scenario

Browser Level Comparison: TruClient (StormRunner) vs. Flood Element (Flood)

StormRunner and Flood are two of the only solutions on the market that offer support for load testing via real browsers.  This technology is great for measuring full stack performance as users would really see it, and is often times the only way to test technologies like AJAX or Single Page Applications.

Looking at the cost of an identical test, which might be a one time test with 5,000 users in TruClient/Flood Element for 1 hour, the costs would be as follows

  • Flood: 5,000 users for 1 hour would require 100 nodes (load injectors) for 1 hour, which comes to a total cost of $499
  • StormRunner: 5,000 users for 1 hour would require 5,000 virtual user hours (GUI), which comes to a total cost of $3,200

Verdict: Flood is 6x less expensive than StormRunner in this Browser Level scenario

Conclusion: Choose the Best Choice For You

Both StormRunner and Flood offer a robust set of features that are great for agile teams looking to do testing in a cloud enabled offering.  Since each team has their own requirements, it is important to compare the different criteria here and asses which solution is right for your use case.  

If you are interested to try out either product, you can trial either product here:

  • StormRunner: free trial that allows for 10 tests with up to 100 concurrent users
  • Flood: Free trial that allows for unlimited tests for up to 5 node hours

Start load testing now

It only takes 30 seconds to create an account, and get access to our free-tier to begin load testing without any risk.

Keep reading: related stories
Return to the Flood Blog