One of our most commonly-asked questions is: "What are the best tools for load testing?" The answer will depend on a few things: your reason for load testing, the application you want to test, what kind of load testing you want to carry out, your team's experience with scripting, and how many virtual users you want to run.
Note: In this video, I say that traditional load testing tools can't simulate requests that are triggered by interactions on the client-side. This isn't technically true: if you know what the requests are, you can build those requests manually on a protocol-level tool (like JMeter or Gatling). However, this takes more work compared to browser-level tools that just do this natively.
Transcript:
Hi everyone, Nicole back again with another Ask a Flooder, and this time I'm tackling one of the questions we get asked A LOT, which is:
What are the best tools for load testing?
Why open-source load testing tools are awesome
Now, I want to preface this by saying that we really love open-source tools here at Flood, because 1) they're really pretty full-fledged these days. You don't have to make any compromises on features, and, in fact, some of these open-source tools have more features than some newer load testing tools just because they've been out longer. And secondly, they have large communities. Because they're open-source, lots of people are using it, and lots of people are developing for it. So there are lots of plugins normally, and they support any kind of protocol that you can think of. And lastly, cost. It's hard not to mention cost because it can be a really important factor depending on the budget of your project. And if you're starting out as a load tester, I think that it might even be the best way to start with load testing, because you can take that knowledge with you and apply it to other projects and other companies regardless of the budget that they have.
Questions to ask yourself in choosing a load testing tool:
Why are you doing load testing?
I always advocate starting with WHY. Why are you actually load testing? Is there a particular production defect that you noticed that you're trying to solve? Is there already a problem, or are you just preparing to launch? These are things that are going to inform the type of tool that you use because you might want to make sure that that tool supports the protocols that you need.
What are you load testing?
Secondly, what type of app are you testing? Because if you have a single-page app, where your end-user is interacting with the page and client-side scripts are running to make requests to servers, then that's something that most traditional load testing tools aren't going to be able to do.
There are two general types of load testing tools: there's browser-based, and there's protocol-based. I'll leave a link in the description talking about the differences between the two, but in general, if you want to go for browser-level testing, I would suggest Selenium and Element. Now, they're both very good options. Selenium is probably the more robust in terms of automation of the two, but Element is better one in terms of efficiency because it was built with performance in mind. So Selenium's automation-first and Element is really performance-first.
Now, for protocol-level tools, you can't go wrong with either JMeter or Gatling.
What experience does your team have with scripting?
Thirdly, what's the level of technical expertise that your team has? I think it's important to be realistic about what your team can do, given the amount of time that they have. So if you have a team that has never done load testing and has never done any scripting before, then I would suggest using our Test Builder or something similar. On Flood, we have a Test Builder that is really just an easy interface that actually runs JMeter under the hood, but it just kind of simplifies JMeter and boils it down to what are the URLs that you want to test. That is a really easy way to get started, and it's also a great [way to run a] proof of concept. If you have some automation testers on your team that are trying to do the load testing, well, maybe you should stick to tools that they already use. So it might be easier for them to get up and running with either Selenium or Element rather than learning a new tool. And if you have developers on your team, you might consider Ruby JMeter rather than the vanilla JMeter because Ruby JMeter is a lot easier to deal with--- because you don't have to deal with the XML files of a JMeter file (a normal JMX script). You can just plan out your load tests in Ruby. Or, Gatling is also a really good option because it allows you to write scripts in Scala. Scala is really really powerful, and you can create an entire framework with Gatling, which is also something to keep in mind.
How many virtual users do you want to run in your load tests?
And lastly, how many users do you want to simulate? Now protocol-level tools are going to really shine here because Element and Selenium both require a higher usage of resources on your load generators just because of the complexities involved in starting a browser instance, whereas protocol-level tools are just super efficient at sending these requests. So I would suggest that if you're talking about thousands of users, I would use the protocol-level tools unless there's a really good reason why that's not going to give you the results that you're looking for. Between JMeter and Gatling-- well, they're both excellent tools, but Gatling has been shown to particularly be good for really high levels of usage. But JMeter is also a good alternative, and you can run thousands of users of JMeter without any problem.
Just get started!
As always, I'd suggest just getting started. Pick whatever tool sounds like it might be the best for you, do a proof of concept on that one, and you can always try the other ones as well. On Flood, we don't restrict you to just using one tool, so you can use one; you can use all four; you can run them at the same time if you want. It all depends on what you need for your testing.
Till next time, happy flooding!
Other resources
Choosing a load testing tool: http://bit.ly/2IIPIRy
Get started with Test Builder: http://bit.ly/38JrT6P
Get started with JMeter: http://bit.ly/39GFcGt
Get started with Ruby JMeter: http://bit.ly/2wUuq0A
Get started with Gatling: http://bit.ly/38Kn7WD
Get started with Element: http://bit.ly/2wSgvIq
Get started with Selenium: http://bit.ly/33eWcBq
JMeter vs Gatling: http://bit.ly/38MvfWO