This is a guest post provided thanks to QASource.
The increasing use of technology has spurred a rapid growth of users, making it crucial for organizations to perform load testing on the application for a massive user load. Launching new software is incredibly stressful- the performance of your software upon launch can make or break your product’s success.
Cloud Load Testing vs. Traditional Load Testing
One of the key challenges for load testing is setting up the load generator infrastructure to test the application with a realistic user simulation. It is not surprising this is a major challenge, since traditional load testing tools might require owning and maintaining dozens of servers. Because of this, many companies will skip out on load testing their software until major events (on-sales, holiday events, etc.) are approaching. When these events get closer on the calendar, thousands or even millions of people start using your product immediately. In theory, that’s ideal, but if your application can’t support a large user load, you may never get those initial users back or prevent them from spreading possible negative word-of-mouth about your brand.
If you plan to perform traditional load testing in-house, you may need to purchase and set up the physical boxes that are required for load generation, which would become a time-consuming, arduous and expensive approach. Cloud load testing can be an easier way to get the servers you need to run your test with less investment and overhead.
In our 17 years of QA testing experience, we have found that Tricentis Flood is the best fit for cloud load and stress testing as it provides all of the necessary features that both we and our clients require. (Tricentis Flood has plenty of success stories to back up this claim as well.)
Requirements for a Cloud Load Testing Tool
Before diving in to using Flood on our first project, we gathered a list of requirements we were looking for in a cloud load testing tool. Flood fit most all of the requirements we were looking for:
- Ability to easily spin up the nodes depending upon the purpose of the load test. While some tests only require a couple of load injectors, many tests may require thousands of servers to simulate the required load.
- Ability to support multiple open-source load testing tools, such as JMeter and Gatling, which we also use to help us execute tests. While some teams prefer JMeter, others like Gatling, and the benefits of each tool are unique.
- Ability to trigger the test from different geo-locations based on test requirements, to simulate the increasingly global nature of user bases.
- Integration capabilities of the cloud load testing tool with an APM tool, such as AppDynamics, Dynatrace, or New Relic, to generate results in a single console.
- Excellent customer support, with the ability to support a global user base across multiple time zones.
- Flexible subscription plans based on usage rather than proprietary and unclear pricing models, with affordable prices per test execution.
After we got started with the Flood testing tool, our engineers quickly learned some best practices for running load tests with Flood.
Best practices for Flood Test Execution
After working with Flood on a few initial projects, we learned the ins and outs of the new tool. The key takeaways to keep in mind when using Flood for cloud load testing are:
- Identify number of users supported by a single node: It is important to identify the amount of users that are supported by a single node without exhausting the resources. This is where good monitoring of load injectors will come into play.
- Parameterize test execution settings: While setting up the test, it’s good to parameterize the test settings in the performance test script in order to avoid frequent changes in the script for every test execution
- Pass parameters in through the Flood UI: Utilize the 'Advanced Parameters' in Flood test to set parameters such as httpclient4.retrycount and -JcookieManager.save.cookies. Flood supports the overriding of user defined variables and uploading of user.properties file.
- Use different test data for different nodes: If the application workflow requires different test data such as username and password, then that needs to be passed separately to each node through .csv or .txt files as node_0.csv and node_1.csv. By using the global Flood environment variables to assign a particular node a fixed csv data file.
- Monitor continuously with Flood timeline: During test execution, it is very important that you monitor the real time behavior of the application in order to determine the application’s capacity.
- Monitor grid resources: For accurate test results, it’s crucial to monitor the grid resources during the test execution to make sure that the resources are not exhausted during a heavy user load.
- Optimally use node hours: The grid should be stopped immediately if the test has completed before the duration that was set for the grid, as this would save the unnecessary consumption of node hours
- Analyze traces and logs: To troubleshoot the root cause of any test set up related issue, the best approach is to download the test results and analyze various log files generated by Flood such as traces.log, Flood.log, and grid.log.
- Integrate with APM tools: It's good practice to integrate Flood with an APM tool such as Dynatrace, New Relic, and AppDynamics to receive the consolidated test results in a single platform.
- Estimate node hours accurately: During the test planning, it’s important to analyze the test iterations and duration so that an appropriate plan can be prescribed based on the requirement. Flood has created a node hour calculator specifically for this use case.
Overall, we found Flood easy to use when implemented properly, without much of any maintenance required on our end to keep tests running smoothly.
About QASource
QASource is an outsourcing software testing company that offers a variety of testing services to clients from startups to Fortune 500 companies. They provide performance testing for an array of different software applications built on different technologies for different industries such as healthcare, legal matter management, secure storage solutions and cloud content management. QAsource provides an automated solution that has the flexibility to meet the client’s requirements in order to test every aspect of their software product, almost always including load testing.
If you’re looking for a team to support you with your load testing and/or other QA testing needs, they’d love to learn more about your company and the project you’re working on: reach out to QASource.
Getting Started with Flood
With these awesome tips from QASource, cloud load testing with Flood should be a breeze. Sign up for a free trial today to run a load test and see how easy it is to share performance insights with your team.
If you have any other helpful tips for cloud load testing with Flood, we’d love to hear from you. Drop us a note and share any ideas that are working for you and feel free to ask our team any of your tough questions!