Welcome!

Linux Containers Authors: Pat Romanski, Elizabeth White, Liz McMillan, Vaibhaw Pandey, Yeshim Deniz

Related Topics: @DevOpsSummit, Microservices Expo, Linux Containers, Containers Expo Blog, @CloudExpo

@DevOpsSummit: Article

Canary Releases | @DevOpsSummit @Anders_Wallgren #DevOps #Microservices

Using Canary Releases and Early Life Support to improve Production Releases

Using Canary Releases and Early Life Support to improve Production Releases

We've all had that moment where a friend comments (or complains) on social media about the new layout of a particular site. But you still see it the way it has been for months, and you have no idea what they are talking about. This is because the company that runs the site is likely using a Canary Release.

This type of release is named after the early 20th century practice where canaries were used as sentinels in coal mines to detect carbon monoxide. According to Wikipedia: "Animal sentinels, or sentinel species, are animals used to detect risks to humans by providing advance warning of a danger".  Thus the canary would get sick from the carbon monoxide before it could seriously affect the men, giving them an early warning sign to be able to evacuate the mine before they, too, became sick.  The Canary Release allows the product release to be deployed to an early warning system, where it can be tested before releasing it across all of the production environments. ITIL refers to these types of releases as Phased Releases.

Canary Releases can be an important tool for a company when undertaking CD or evenContinuous Deployment.  By rolling out new features quickly, and testing them with live users, feedback can be implemented quickly and efficiently. Testing with actual users in a production environment also allows the company to gain insight into actual performance numbers, rather than just in a simulated testing environment.

As Jez Humble states, one of the principles of low-risk releases is to have incremental releases. "With this pattern, rather than upgrading a whole cluster to the latest version all at once, you do it incrementally." Canary Releases allow you to incrementally install and test new releases, without risking your entire production environment.

How to implement Canary Releases
To implement Canary Releases, it is imperative that the deployments be a) automated, and b) the configurations in production be set up in such a way that releasing to a limited number of production nodes is possible. Your deployment automation should be configurable in such a way that the application can be deployed to a single set of servers. Using a deployment automation solution, like ElectricFlow Deploy, will allow you to select the environments on which to deploy your software release, and then scale from there, if everything looks good.

A Canary Release will set up a small number of environments to test in production, before rolling out the release to all of the production nodes. In this way, the sentinel release environment(s) serves to allow the team to watch over the release until the system can be brought up to a capacity that will match production.  The environments used to test the production release can be taken out of the production server rotation (remove them from the load balancer), have the new version installed, then traffic can be routed to them in a controlled manner, allowing the testing to be managed, and brought up to production activity levels slowly. If the tests are successful, other environments can be removed from the load balancer and have the new version installed until all systems are made consistent with the latest release.

If any problems are found during the Canary Release, since it is a limited number of environments which have received the new release, it will be easier to revert those environments to the previous release than it would have been if all of the production environments were involved.  To revert, these Canary environments can be taken out of the server rotation again, and then returned once they are reverted back to the last working version of the product. This type of release ensures that not all customers are affected if there were problems with the new released version.

Early Life Support
Another factor to improve the Release Process is to use an Early Life Support (ELS) process. In this instance, a Deployment Team is organized, with well defined and documented criteria on what constitutes an acceptable release. The team monitors the release for a specified amount of time, determined by the company, to ensure that it meets the exit criteria for the release.  The ITIL Service Transition standards have a good description of the ELS Deployment Team. "The deployment team's aim is to stabilize the service for the target deployment group or environment as quickly and effectively as possible."

With the ELS Deployment Team in place, the company can closely monitor the release, until the exit criteria are met and the release is considered stable and ready to run without monitoring. The Deployment Team is another early-warning device that allows for a quick reaction team to respond to any issues found in production. While this may not be as effective in an Agile environment, when multiple releases can occur in a single day, it can be very effective in larger companies, where resources are more available and larger releases are done less frequently.

Using a combination of Canary Releases and Early Life Support, companies can work to find the right balance to ensure that their Product Releases are stable and ready for production, improving the overall customer experience for their users.

So now you can tell your friends that this is how they might still be seeing the old Facebook interface, for example, while others may be commenting about their experiences on the new interface - since their instance is served from the Canary server.

Want More?
Canary Releases are just one way to test your deployments and ensure a successful release into Production. Other patterns are Blue/Green deployments, Feature-flag, Dark Launches, Environment Promotion, and more. To learn about other common patterns to deploying mission-critical apps, the differences between them, and what does each mean in terms of your architecture, code base or Operations - check out the video replay of one of our recent #c9d9 episodes below:

Continuous Discussions: Episode 25 Strategies for Deploying Mission Critical Apps at Scale

More Stories By Anders Wallgren

Anders Wallgren is Chief Technology Officer of Electric Cloud. Anders brings with him over 25 years of in-depth experience designing and building commercial software. Prior to joining Electric Cloud, Anders held executive positions at Aceva, Archistra, and Impresse. Anders also held management positions at Macromedia (MACR), Common Ground Software and Verity (VRTY), where he played critical technical leadership roles in delivering award winning technologies such as Macromedia’s Director 7 and various Shockwave products.

@ThingsExpo Stories
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of the 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to gre...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and shared the must-have mindsets for removing complexity from the develop...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
"Evatronix provides design services to companies that need to integrate the IoT technology in their products but they don't necessarily have the expertise, knowledge and design team to do so," explained Adam Morawiec, VP of Business Development at Evatronix, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...