Welcome!

Linux Containers Authors: Jignesh Solanki, Yeshim Deniz, Karthick Viswanathan, Pat Romanski, Mehdi Daoudi

Related Topics: Microservices Expo

Microservices Expo: Article

Load Testing Web Services

Software testing is crucial to SDLC and load testing is integral to any efficient testing scheme

The quality of any application is determined by the robustness and scalability of the system. It's mandatory to simulate the actual environment and test the application for preparedness. Web Services-savvy applications need a different methodology for testing in a real-world scenario. The UI-less nature of Web Services presents a significant challenge in testing such applications. The whole persona of consumer stubs with different payloads dictates the planning of Web Services load-testing schemes. This paper talks about the different aspects of load testing and areas of contention that need special attention. This will be helpful in not only building a better application but also compiling a robust, high-quality enterprise architecture.

Web Services are the natural delivery mechanism to achieve SOA. While having the potential to free enterprises from the endless cycle of vendor-specific hardware/software upgrades by ensuring interoperability, they bring in integration complexities and the overhead of maintaining compatibility with the underlying EIS applications/systems. This brings in an absolutely different perspective to testing Web Services.

Web Services applications generally use a lot of data transformation, wraparounds (wrappers), translation, and abstraction to bring about the promised interoperability and portability. Their dependence on bandwidth-heavy protocols like SOAP doesn't ensure many performance benefits when compared to legacy applications (which tend to be very tightly coupled). Parameters like response time, throughput, and CPU utilization for transactions determine the viability of a real-world business application. Extensive testing of Web Services based on these parameters brings to the fore the most common performance constraints associated with them. The test results not only indicate whether the associated benchmarks are attained, but also if the service can scale to meet demands imposed by concurrent access from multiple users, simulated or otherwise.

Web Service endpoints generally also have very high visibility. They have to service multiple clients over the network simultaneously, maintaining robustness and availability at the same time. In such a situation, performance becomes even more crucial. Thus, the significance of proper performance testing for Web Services can't be overemphasized.

A Web Service, like any other application, can be subject to a wide range of test conditions and testing strategies. Some of them being functional testing, regression testing, performance testing, stress testing, and load testing. This paper will focus only on the load testing of Web Services. The expected behavior of a Web Service will be evaluated against various performance criteria when concurrent access by multiple clients is simulated. It becomes crucial to ensure that apart from optimizing design and implementation, Web Services have to be tested for throughput, efficiency, and response simulating real-world conditions as closely as possible. This is where load testing plays a major role. A properly designed load-testing strategy can simulate real-world load and performance scenarios with minimal hassle and cost. User loads and network conditions of varying nature can be effortlessly created and replicated. Testing can be undertaken till the output charts show a performance range considered acceptable for an application of its nature. Load-testing results can hence be taken as a strong indicator of application performance in actual business environments.

To ensure optimal testing of Web Services, the test cases have been designed keeping the following parameters in mind:

  • Size of payload: This tests the amount or size of the incoming requests. This parameter is vital in determining the threshold of data beyond which the service behaves in unexpected ways.
  • Concurrency: The test cases have to simulate simultaneous access of the service by multiple clients to replicate real-world conditions.
  • Latency: It can be defined as the time from issuing a request to the service from the client and the receipt of the first response. This parameter encapsulates the performance of the service, bandwidth in the network, and other communication overheads. It's important that latency be minimized as far as possible, at least up to a tolerable point.
  • System utilization: The net CPU and memory resources consumed by the service under varying loads in normal enterprise environments should be captured by the load-testing scheme. This helps in identifying potential bottlenecks and points out areas of improvement.
These parameters shall be discussed in detail later:

Load Testing with Reference to Web Services
Load testing of Web Services is significantly different from testing of other applications since their performance is not just attributed to how robust the underlying architecture is but also to the network overheads, underlying processing involved, and the performance of the Web server that hosts the service. The behavior of the SOAP engine also invariably adds to the architecture of service provider systems. Certain major areas of contention when evaluating the Web Service performance that will be discussed here are:

  • The impact of an incremental XML payload size wrapped inside a SOAP message
  • The impact of a chosen style/use during the design of a Web Service
  • The serialization/de-serialization involved in processing the SOAP message
  • The underlying parsing model and validation schemes chosen to process the XML payload
The results of the load testing such as response time graphs will further depict the vitality of the load testing of Web Services to ascertain their conformity prior to actual deployment to enable their wide-scale adoption without compromising their performance and scalability characteristics, enabling the enforcement of stringent operational, behavioral, and non-functional requirements that are inherent in the successful realization of any business process.

Load Testing Metrics and Parameters
The results obtained by load testing Web Services can potentially be reflected in terms of the following parameters.

  • Response time: It's the most important parameter to reflect the quality of a Web Service. Response time is the total time it takes after the client sends a request till it gets a response. This includes the time the message remains in transit on the network, which can't be measured exclusively by any load-testing tool. So we're restricted to testing Web Services deployed on a local machine. The result will be a graph measuring the average response time against the number of virtual users.
  • Number of transactions passed/failed: This parameter simply shows the total number of transactions passed or failed.
  • Throughput: It's measured in bytes and represents the amount of data that the virtual users receive from the server at any given second. We can compare this graph to the response-time graph to see how the throughput affects transaction performance.
  • Load size: The number of concurrent virtual users trying to access the Web Service at any particular instance in an interval of time.
  • CPU utilization: The amount of CPU time used by the Web Service while processing the request.
  • Memory utilization: The amount of memory used by the Web Service while processing the request.
  • Wait Time (Average Latency): The time it takes from when a request is sent until the first byte is received.
Performance Bottlenecks & Areas of Contention
Web Services are simply components deployed on a server. Most of the Web Services today are exposed out of existing components such as Enterprise Java Beans. Hence, in theory, we should be able to use the existing testing mechanisms and performance-enhancing strategies. But as already discussed load testing Web Services is quite different. The performance of Web Services is influenced by a lot of factors like bottlenecks in the network, processing at intermediate nodes if any, pre-processing of the SOAP message at the SOAP engine before it's dispatched to the service, etc. To identify the areas of contention, we'll look first at the architecture of the SOAP message processing on the service side. (see Figure 1)

A client application creates a SOAP message containing the XML payload, which can be either a SOAP-RPC-encoded request or a document-style message. The client sends this message along with the service endpoint URL to the SOAP client runtime, which in turn sends it over the network. Once the SOAP message is delivered to the SOAP runtime at the service, it passes through handlers (if any) that handle the processing of any additional tags for WS-Security, WS-Addressing, etc. Then the SOAP runtime converts the XML message into programming language-specific objects if required by the application. The Web Service processes the request message and formulates a response. The SOAP runtime on the service side takes care of creating a SOAP message and dispatching it back to the client.

So, apart from the actual processing of the Web Service, there's some additional processing involved before and after the Web Service builds a response. Let's identify the bottlenecks involved in invoking a Web Service:

  • XML processing and overheads: Since XML data is verbose and contains lot of metadata information, processing of XML is a major performance bottleneck. Processing XML involves parsing, validating the data against schema, and marshalling/unmarshalling. It's quite memory- and CPU-intensive and the response time takes a hit if the proper strategies aren't followed.
  • Parsing of XML: The larger the SOAP message, the longer it takes to parse it. Parsing SOAP messages is a major contributor to performance issues with Web Services. A memory-efficient parser like StAX (a pull parser) should be used in place of a memory-intensive parser like DOM.
  • Serialization/de-serialization: When the SOAP engine on the service side receives a SOAP request, it de-serializes the XML data according to the encoding format mentioned, i.e., extracts the payload out of it, and creates objects that are used by the Web Service. After the Web Service executes the business logic, the SOAP engine takes care of serializing the response back to XML and sends the data to the client. For huge XML documents, the serialization/de-serialization takes a performance hit if a proper mechanism isn't selected.
  • Select a proper style for your Web Service: The two most pre-dominantly used styles of Web Services are document/literal and RPC/encoded. The SOAP message of the RPC-encoded style of Web Service contains the type-encoding information, which is an overhead on the SOAP engine and degrades the throughput performance whereas the document/literal SOAP message contains no such type-encoding information. The XML payload can be easily validated against the schema included in the WSDL. Also, the data binding specific to a SOAP engine can be switched off in the case of a document/literal-style Web Service. This enables one to use a custom binding framework like XMLBeans, castor, JAXB, etc. This is especially useful when the application uses a large number of complex custom data types.
  • Processing by handlers: The SOAP engine first dispatches the SOAP message to the handlers. The handlers may be responsible for performing additional processing like authentication, encryption/decryption of the XML payload, parsing the SOAP message for any information like WS-Security, WS-Addressing, etc.

More Stories By Bijoy Majumdar

Bijoy Majumdar is a member of the Web Services COE (Center of Excellence) for Infosys Technologies, a global IT consulting firm, and has substantial experience in publishing papers, presenting papers at conferences, and defining standards for SOA and Web services. Prior to Infosys, Bijoy Majumdar worked as an IT Analyst, and had been a member of the GE Center of Excellence (e-center) under the E-Business Practice of Tata Consultancy Services.

More Stories By Ujval Mysore

Ujval Mysore is a member of the Web Services COE (Center of Excellence) for Infosys Tehcnologies, a global IT consulting firm, and have substantial experience in publishing papers, presenting papers at conferences, and defining standards for SOA and Web services. The Web Services COE specializes in SOA, Web services, and other related technologies. Dr. Srinivas Padmanabhuni heads the Web Services COE.

More Stories By Lipika Sahoo

Lipika Sahoo currently works with the Web Services Centre of Excellence in SETLabs, the technology research division at Infosys Technologies, India. Her core area of work involves dynamic adaptability and management of Web services. She is currently involved in various research activities within the group relating to MDA and AOSD.

More Stories By Sunny Saxena

Sunny Saxena currently works with the Web Services Centre of Excellence in SETLabs, the technology research division at Infosys Technologies, India. His interests range from Web service security platforms to aspect-oriented development models.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
JDJ News Desk 10/19/06 12:12:22 PM EDT

The quality of any application is determined by the robustness and scalability of the system. It's mandatory to simulate the actual environment and test the application for preparedness. Web Services-savvy applications need a different methodology for testing in a real-world scenario. The UI-less nature of Web Services presents a significant challenge in testing such applications. The whole persona of consumer stubs with different payloads dictates the planning of Web Services load-testing schemes. This paper talks about the different aspects of load testing and areas of contention that need special attention. This will be helpful in not only building a better application but also compiling a robust, high-quality enterprise architecture.

@ThingsExpo Stories
Product connectivity goes hand and hand these days with increased use of personal data. New IoT devices are becoming more personalized than ever before. In his session at 22nd Cloud Expo | DXWorld Expo, Nicolas Fierro, CEO of MIMIR Blockchain Solutions, will discuss how in order to protect your data and privacy, IoT applications need to embrace Blockchain technology for a new level of product security never before seen - or needed.
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settle...
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
Imagine if you will, a retail floor so densely packed with sensors that they can pick up the movements of insects scurrying across a store aisle. Or a component of a piece of factory equipment so well-instrumented that its digital twin provides resolution down to the micrometer.
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
We are given a desktop platform with Java 8 or Java 9 installed and seek to find a way to deploy high-performance Java applications that use Java 3D and/or Jogl without having to run an installer. We are subject to the constraint that the applications be signed and deployed so that they can be run in a trusted environment (i.e., outside of the sandbox). Further, we seek to do this in a way that does not depend on bundling a JRE with our applications, as this makes downloads and installations rat...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
DX World EXPO, LLC, a Lighthouse Point, Florida-based startup trade show producer and the creator of "DXWorldEXPO® - Digital Transformation Conference & Expo" has announced its executive management team. The team is headed by Levent Selamoglu, who has been named CEO. "Now is the time for a truly global DX event, to bring together the leading minds from the technology world in a conversation about Digital Transformation," he said in making the announcement.
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lead...
Digital Transformation (DX) is not a "one-size-fits all" strategy. Each organization needs to develop its own unique, long-term DX plan. It must do so by realizing that we now live in a data-driven age, and that technologies such as Cloud Computing, Big Data, the IoT, Cognitive Computing, and Blockchain are only tools. In her general session at 21st Cloud Expo, Rebecca Wanta explained how the strategy must focus on DX and include a commitment from top management to create great IT jobs, monitor ...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
The IoT Will Grow: In what might be the most obvious prediction of the decade, the IoT will continue to expand next year, with more and more devices coming online every single day. What isn’t so obvious about this prediction: where that growth will occur. The retail, healthcare, and industrial/supply chain industries will likely see the greatest growth. Forrester Research has predicted the IoT will become “the backbone” of customer value as it continues to grow. It is no surprise that retail is ...
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of the 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to gre...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and shared the must-have mindsets for removing complexity from the develop...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
"Evatronix provides design services to companies that need to integrate the IoT technology in their products but they don't necessarily have the expertise, knowledge and design team to do so," explained Adam Morawiec, VP of Business Development at Evatronix, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...