Welcome!

Linux Containers Authors: Elizabeth White, Pat Romanski, Liz McMillan, Stackify Blog, Yeshim Deniz

Related Topics: @DevOpsSummit, Linux Containers, Containers Expo Blog

@DevOpsSummit: Blog Feed Post

Putting the Test Back into DevOps | @DevOpsSummit #DevOps #BigData #Microservices

DevOps has moved from being something based on speculation about what development could be, to a real part of the developer role

Putting the Test Back into DevOps
by Justin Rohrman

DevOps has become synonymous with empowering developers to move faster and deliver more software, but has unintentionally moved software quality and testing into a corner. The pendulum that swings back and forth between very slow testing and staying far away from overly technical solutions has swung hard with DevOps, completely ignoring the value that testers can bring by forcing customers to test new code and creating teams that rely monitoring systems to find things that could have been caught before customers noticed.

DevOps has moved from being something based on speculation about what development could be, to a real part of the developer role. Let's see how we can swing this trend back to a more responsible place, still using DevOps to release software faster but with a level of quality that customers will be happy with.

Continuous Integration and Delivery
Continuous Integration
(CI) and Delivery (CD) are two of the most fundamental concepts built into DevOps. They have helped us talk about tools that will build new products every time a new line of code is added to the repository and then as soon as that build is done, deploy it to production. Some of the more technology focused companies like Github have pushed this idea as far as it can go and are deploying new software to production many times every day.

The side effect of this style of fast paced development and release is that any time for a real live person to try the software out before delivering is squeezed out. As soon as there is software that can run, it goes into production. Paying customers are the new testers, and monitoring and reporting systems are the new bug reports.

If we dial this back a little bit, we get a strategy where developers do CI on their own environment, getting a new build every time they commit to their local source code repository. And after checking in to the main repo, code is continuously deployed to a staging environment.

I have had a lot of success with continuous deployment to a headless API server on smaller teams. One team I worked on had two people working on the API platform that the rest of our product was built on. Each time they would commit code, a new build would go to that server. Normally before that would happen, we would sit together and talk about the changes and their concerns. I might start stubbing out a few automated checks, and writing down some test ideas.

We were still using DevOps concepts to build an API quickly, but we were doing it in a way that didn't force customers to deal with our problems.

Monitoring Is Only Part of the Solution
A few large companies are using complex API and Web monitoring systems running in the background, sucking up large amounts of log data from the API looking for a few important keywords like Exception, and Error. Every time these words are found, emails and text messages are sent to let the development staff know that something has gone wrong.

Maybe you have a completely componentized product where you can flip bits to turn features on and off quickly to reduce exposure to faults. Some companies have bits and pieces of product that are built as components, a handful have a large amount. Most have little or none. For smaller companies, this would mean spending equal amounts of time developing architecture as on building product. That isn't a ratio most founders and investors would want to see.

Monitoring and rollback systems are great for API products, but are usually not feasible for young companies that urgently need product to sell.

Landmines for Consumers and Holes in Your Revenue
Using monitoring without pre-production testing is dangerous for you and your consumers. Imagine you are releasing a new version of your API that will help customers to use Geolocation information on their phone to find stores nearby. The code under the hood of the API has some automated checks using faked data to see if stores were returned correctly, but there are only a few and they are pretty simple.

One of your first customers to use this feature live in the island paradise of Hawaii and it just so happens that there are lots of cities there with special characters in the name. A few hours after deploy, the server log files are blowing up with errors from people in Kāne‘ohe and ‘Ewa Gentry trying to find the closest Dunkin Donuts.

Emails start flying through the development office and the feature gets shut off minutes later, but the damage is done. By the time your support person gets in touch with a developer, and that developer investigates and takes action, your customers have given up. About half of those users uninstalled the app and are searching with Google instead.

"Testing new API changes can expose your product to black swan problems in a way that DevOps never will."

There are real consequences for using your customers to test new code. Placing a skilled tester in front of that API probably would have brought up questions like "What happens for cities with really long names, or special characters, or very small cities that might not be in all mapping systems". Testing new API changes can expose your product to black swan problems in a way that DevOps never will.

Cases like this actually happen and are good examples of where using DevOps concepts to deliver internally, rather than to production, would have saved a few customers and company money.

DevOps presents a powerful set of ideas that can help to deliver code to customers faster than we used to. They can also be a dangerous way to deliver bad code and buggy product much faster than we would like to. If we slow down a little bit by pairing DevOps themes with skilled testers, these ideas and tools can help to deliver software and API updates faster than would be possible without them without exposing our customers to new kinds of risk.

Do you have experience integrating DevOps with testing? We would love to hear your story.

Read the original blog entry...

More Stories By SmartBear Blog

As the leader in software quality tools for the connected world, SmartBear supports more than two million software professionals and over 25,000 organizations in 90 countries that use its products to build and deliver the world’s greatest applications. With today’s applications deploying on mobile, Web, desktop, Internet of Things (IoT) or even embedded computing platforms, the connected nature of these applications through public and private APIs presents a unique set of challenges for developers, testers and operations teams. SmartBear's software quality tools assist with code review, functional and load testing, API readiness as well as performance monitoring of these modern applications.

@ThingsExpo Stories
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
DXWorldEXPO LLC announced today that the upcoming DXWorldEXPO | CloudEXPO New York event will feature 10 companies from Poland to participate at the "Poland Digital Transformation Pavilion" on November 12-13, 2018.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
The best way to leverage your CloudEXPO | DXWorldEXPO presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering CloudEXPO | DXWorldEXPO will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at CloudEXPO. Product announcements during our show provide your company with the most reach through our targeted audienc...
JETRO showcased Japan Digital Transformation Pavilion at SYS-CON's 21st International Cloud Expo® at the Santa Clara Convention Center in Santa Clara, CA. The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get...
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. All In Mobile is a mobile app development company from Poland. Since 2014, they maintain passion for developing mobile applications for enterprises and startups worldwide.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world.
As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and ...
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
"We are a well-established player in the application life cycle management market and we also have a very strong version control product," stated Flint Brenton, CEO of CollabNet,, in this SYS-CON.tv interview at 18th Cloud Expo at the Javits Center in New York City, NY.
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
Most people haven’t heard the word, “gamification,” even though they probably, and perhaps unwittingly, participate in it every day. Gamification is “the process of adding games or game-like elements to something (as a task) so as to encourage participation.” Further, gamification is about bringing game mechanics – rules, constructs, processes, and methods – into the real world in an effort to engage people. In his session at @ThingsExpo, Robert Endo, owner and engagement manager of Intrepid D...
Recently, WebRTC has a lot of eyes from market. The use cases of WebRTC are expanding - video chat, online education, online health care etc. Not only for human-to-human communication, but also IoT use cases such as machine to human use cases can be seen recently. One of the typical use-case is remote camera monitoring. With WebRTC, people can have interoperability and flexibility for deploying monitoring service. However, the benefit of WebRTC for IoT is not only its convenience and interopera...