Scalability Factors of JMeter In Performance Testing Projects

Abstract: Testing web applications for performance is fundamentally different and complex than testing them for functional correctness. Various flavors of load generation tools are available to simulate the expected load levels on servers, network or web application to test their ability to sustain concurrent heavy load under realistic conditions. In order to carry out effective performance testing of web applications one has to ensure that sufficiently powerful hardware is used to generate required load levels. At the same time, one would prefer to avoid investing in unnecessarily expensive hardware “just to be sure”. Having an effective model for estimating the load generation capabilities of load generation tools on different hardware configurations can greatly help in taking care of both these requirements.

Unfortunately, the only such model we currently have is the simplistic set of benchmarks offered by different Commercial Performance test tool vendors.  These benchmarks spell out how many “virtual users” their tool can simulate on different hardware configurations.  In fact the load generation capability of any tool is a function of multiple factors and not just the underlying hardware configuration. Hence, the simplistic model is therefore not usable in practice.  Some of the other factors that affect the load generation capacity of a tool include application response sizes, application response times and the complexity of client-side activity.

The Performance Engineering group from Aztecsoft’s Expert Services Group attempted to create such a model for Performance testing tool “JMeter”. We have created an experimental setup wherein we measured the sensitivity of JMeter performance to some of the factors described above and used this data to construct the model. This paper describes our experimental setup and presents our model for determining load generation capabilities of JMeter.


Profile: Budhaditya Das is presently working as a Test Analyst at Aztecsoft iTest Expert Services Group (Performance Testing). As a member of this group he is responsible for R&D, tool development, pre-sales and training in Performance Testing. In addition his areas of interest also include Web Application Security testing and test automation.
He has Bachelor’s degree in Electronics from Shivaji University, India. He has 3.5 years of experience in the software industry.


Profile: Priyanka Mane is presently working at Aztecsoft iTest and is responsible for R&D activities in the area of Web Application Performance Testing.
She has Bachelor’s degree in Computer Science and Engineering from Shivaji University, India.