Welcome!

Linux Containers Authors: Liz McMillan, Pat Romanski, Elizabeth White, Yeshim Deniz, Flint Brenton

Related Topics: @DevOpsSummit, Linux Containers, Containers Expo Blog

@DevOpsSummit: Article

Continuous Performance Validation | @CloudExpo #DevOps #IoT #BigData

Today, user experience is crucial to an application's success

Achieving Continuous Performance Validation with Synthetic Users

In most modern development groups, there's a big focus on creating code that works. The idea goes that in more traditional, waterfall approaches, errors in code that aren't fixed until late in the cycle are much more expensive to resolve due to unintended consequences. Today, some groups practice test-driven development as a way to ensure that code is always functional while others work with short agile sprints where all code produced must be usable in the field by the end of each sprint.

There's a common understanding about what it means for a coding task to be "done." Yet, often this "doneness" is only a measure of functionality - not necessarily usability.

Today, user experience is crucial to an application's success, and that goes well beyond what color your button is or how prominently a call-to-action is placed. Users leave your site if pages don't load fast enough or if the site simply feels sluggish when compared with your competitors' sites.

The question is: is there a process for validating that your app is performing up to your customers' expectations and have that process treat performance testing with the same amount of rigor as functional testing?

Yes. It's called Continuous Performance Validation.

Defining Continuous Performance Validation
Continuous Performance Validation is the process of continuously testing, monitoring and improving performance at every stage of the application development lifecycle, from development to production, utilizing automated and collaborative tooling.

The heart of this process is a set of performance test scenarios that sit alongside more traditional functional unit tests. These tests, in conjunction with performance SLAs that can be placed on the task board as inputs to the development process itself, give you a library of performance unit tests that you can mix and match in a number of interesting ways.

As the building blocks of Continuous Performance Validation, these elements let you form long chains of specific behavior to use as test scenarios, mimicking how users actually use your app. These transactions should be used in every phase of the application development lifecycle - development, pre-production, and production - to proactively test and monitor performance across environments.

Before code is released, these scenarios can be pieced together to analyze how load and stress impact your app in different ways. That information can be fed to Operations for infrastructure and capacity planning purposes. Meanwhile, in production, those same scenarios can be executed to see if performance trends in reality match what was predicted in test. Problems can be identified, sent back to dev & test so that issues can be patched quickly.

Synthetic Users: The Key to Continuity
Continuous Performance Validation depends on the idea of creating synthetic users that can move through the app just like real users. Throughout the synthetic users' journeys, they gather metrics on performance and the end user experience.

As part of your continuous integration process, you can continually validate the overall performance of the system by building automated scenarios and plugging them into various stages of the dev process, executed by synthetic users.

In development, synthetic users execute transactions through unit tests that can validate localized performance requirements of key systems. They can fulfill a pre-planned test requirement, similar to TDD, and they can protect against regressions as code changes.

In pre-production, synthetic users can be thrown out from the cloud for truly distributed and georealistic load testing across the entire system. These users traverse various paths through the application and help to define the scalability and geographical contours of your application's distributed user base.

In production, synthetic users operate alongside real users. They navigate the system and exercise the same user paths you have already tested. But instead of sending out millions of them, you only need to send out a few, which gather metrics about performance and availability at particular moments in time and tell you if there is going to be a problem. Synthetic users validate that your performance is up to par and give you insight as to where it is not.

Conclusion
Continuous Performance Validation is a powerful process for apps that need to perform. By putting it in place, you won't be treating performance as an afterthought. Performance testing will be directly integrated into your standard process of development, making performance part of everyone's job description and everyone's responsibility.

Want to learn more? Check out our webinar on Continuous Performance Validation in Agile Development.

Photo Credit: Bicentennial Man (1999) from 1492 Pictures, Columbia Pictures Corporation, Laurence Mark Productions

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

@ThingsExpo Stories
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
Michael Maximilien, better known as max or Dr. Max, is a computer scientist with IBM. At IBM Research Triangle Park, he was a principal engineer for the worldwide industry point-of-sale standard: JavaPOS. At IBM Research, some highlights include pioneering research on semantic Web services, mashups, and cloud computing, and platform-as-a-service. He joined the IBM Cloud Labs in 2014 and works closely with Pivotal Inc., to help make the Cloud Found the best PaaS.
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and ...
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
From 2013, NTT Communications has been providing cPaaS service, SkyWay. Its customer’s expectations for leveraging WebRTC technology are not only typical real-time communication use cases such as Web conference, remote education, but also IoT use cases such as remote camera monitoring, smart-glass, and robotic. Because of this, NTT Communications has numerous IoT business use-cases that its customers are developing on top of PaaS. WebRTC will lead IoT businesses to be more innovative and address...
Rodrigo Coutinho is part of OutSystems' founders' team and currently the Head of Product Design. He provides a cross-functional role where he supports Product Management in defining the positioning and direction of the Agile Platform, while at the same time promoting model-based development and new techniques to deliver applications in the cloud.
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
In his session at Cloud Expo, Alan Winters, U.S. Head of Business Development at MobiDev, presented a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to maximize project result...
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
Personalization has long been the holy grail of marketing. Simply stated, communicate the most relevant offer to the right person and you will increase sales. To achieve this, you must understand the individual. Consequently, digital marketers developed many ways to gather and leverage customer information to deliver targeted experiences. In his session at @ThingsExpo, Lou Casal, Founder and Principal Consultant at Practicala, discussed how the Internet of Things (IoT) has accelerated our abilit...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, discussed the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
No hype cycles or predictions of zillions of things here. IoT is big. You get it. You know your business and have great ideas for a business transformation strategy. What comes next? Time to make it happen. In his session at @ThingsExpo, Jay Mason, Associate Partner at M&S Consulting, presented a step-by-step plan to develop your technology implementation strategy. He discussed the evaluation of communication standards and IoT messaging protocols, data analytics considerations, edge-to-cloud tec...
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settl...
In his session at @ThingsExpo, Dr. Robert Cohen, an economist and senior fellow at the Economic Strategy Institute, presented the findings of a series of six detailed case studies of how large corporations are implementing IoT. The session explored how IoT has improved their economic performance, had major impacts on business models and resulted in impressive ROIs. The companies covered span manufacturing and services firms. He also explored servicification, how manufacturing firms shift from se...