In this Ask a Flooder, Nicole tackles a question we get all the time from users regarding results, particularly for cloud load testing. Sometimes you might notice that response times that are reported in by your load testing tool do not match up with the response time you see from your own browser. What causes this, and how can we fix it?
Transcript:
It's always confusing when your load testing tool reports higher average response times than what you can see while you're manually accessing your application on your browser. There are many things that might cause this, but one of the most common causes for it is that you're overloading your node generators.
As a general rule of thumb, you'll want to keep your CPU utilization at 80% or less throughout the duration of the test. It is normal for it to spike in the beginning, but it should ideally come to a more stable spot below 80%. If your node's CPU utilization is going over 80% for most of the test, then what that means is that your node is struggling to get those requests out, which may lead to some inaccurate results.
You can confirm whether or not this is the cause of your high response times by checking the logs and checking the resource utilization on your nodes.If you're running on Flood, you can do this while the flood is running by going over to Grids, selecting the grid, and then scrolling down on the panel on the right to see the CPU and memory as well as the network throughput.
There are a couple of things you can do to fix this. The first one is to increase any sort of delays or think times in your script — and, if you don't already have any, you should add some. Think times make the script a lot more realistic and a lot closer to what real users would do. If these values are too low, then your load testing tool is just going to run the script as fast as possible, consuming more resources than necessary.
The second thing you can do is reduce the number of users you're running per node. This will free up more resources per node and make it a lot easier to scale up your load tests more effectively.
---
Check out this article for more detailed information on resource over-utilization and other possible reasons for differences between response times in a test and on a browser.