Welcome!

Linux Containers Authors: Zakia Bouachraoui, Yeshim Deniz, Elizabeth White, Liz McMillan, Pat Romanski

Related Topics: @DevOpsSummit, Microservices Expo, Linux Containers, Containers Expo Blog, @CloudExpo

@DevOpsSummit: Article

Canary Releases | @DevOpsSummit @Anders_Wallgren #DevOps #Microservices

Using Canary Releases and Early Life Support to improve Production Releases

Using Canary Releases and Early Life Support to improve Production Releases

We've all had that moment where a friend comments (or complains) on social media about the new layout of a particular site. But you still see it the way it has been for months, and you have no idea what they are talking about. This is because the company that runs the site is likely using a Canary Release.

This type of release is named after the early 20th century practice where canaries were used as sentinels in coal mines to detect carbon monoxide. According to Wikipedia: "Animal sentinels, or sentinel species, are animals used to detect risks to humans by providing advance warning of a danger".  Thus the canary would get sick from the carbon monoxide before it could seriously affect the men, giving them an early warning sign to be able to evacuate the mine before they, too, became sick.  The Canary Release allows the product release to be deployed to an early warning system, where it can be tested before releasing it across all of the production environments. ITIL refers to these types of releases as Phased Releases.

Canary Releases can be an important tool for a company when undertaking CD or evenContinuous Deployment.  By rolling out new features quickly, and testing them with live users, feedback can be implemented quickly and efficiently. Testing with actual users in a production environment also allows the company to gain insight into actual performance numbers, rather than just in a simulated testing environment.

As Jez Humble states, one of the principles of low-risk releases is to have incremental releases. "With this pattern, rather than upgrading a whole cluster to the latest version all at once, you do it incrementally." Canary Releases allow you to incrementally install and test new releases, without risking your entire production environment.

How to implement Canary Releases
To implement Canary Releases, it is imperative that the deployments be a) automated, and b) the configurations in production be set up in such a way that releasing to a limited number of production nodes is possible. Your deployment automation should be configurable in such a way that the application can be deployed to a single set of servers. Using a deployment automation solution, like ElectricFlow Deploy, will allow you to select the environments on which to deploy your software release, and then scale from there, if everything looks good.

A Canary Release will set up a small number of environments to test in production, before rolling out the release to all of the production nodes. In this way, the sentinel release environment(s) serves to allow the team to watch over the release until the system can be brought up to a capacity that will match production.  The environments used to test the production release can be taken out of the production server rotation (remove them from the load balancer), have the new version installed, then traffic can be routed to them in a controlled manner, allowing the testing to be managed, and brought up to production activity levels slowly. If the tests are successful, other environments can be removed from the load balancer and have the new version installed until all systems are made consistent with the latest release.

If any problems are found during the Canary Release, since it is a limited number of environments which have received the new release, it will be easier to revert those environments to the previous release than it would have been if all of the production environments were involved.  To revert, these Canary environments can be taken out of the server rotation again, and then returned once they are reverted back to the last working version of the product. This type of release ensures that not all customers are affected if there were problems with the new released version.

Early Life Support
Another factor to improve the Release Process is to use an Early Life Support (ELS) process. In this instance, a Deployment Team is organized, with well defined and documented criteria on what constitutes an acceptable release. The team monitors the release for a specified amount of time, determined by the company, to ensure that it meets the exit criteria for the release.  The ITIL Service Transition standards have a good description of the ELS Deployment Team. "The deployment team's aim is to stabilize the service for the target deployment group or environment as quickly and effectively as possible."

With the ELS Deployment Team in place, the company can closely monitor the release, until the exit criteria are met and the release is considered stable and ready to run without monitoring. The Deployment Team is another early-warning device that allows for a quick reaction team to respond to any issues found in production. While this may not be as effective in an Agile environment, when multiple releases can occur in a single day, it can be very effective in larger companies, where resources are more available and larger releases are done less frequently.

Using a combination of Canary Releases and Early Life Support, companies can work to find the right balance to ensure that their Product Releases are stable and ready for production, improving the overall customer experience for their users.

So now you can tell your friends that this is how they might still be seeing the old Facebook interface, for example, while others may be commenting about their experiences on the new interface - since their instance is served from the Canary server.

Want More?
Canary Releases are just one way to test your deployments and ensure a successful release into Production. Other patterns are Blue/Green deployments, Feature-flag, Dark Launches, Environment Promotion, and more. To learn about other common patterns to deploying mission-critical apps, the differences between them, and what does each mean in terms of your architecture, code base or Operations - check out the video replay of one of our recent #c9d9 episodes below:

Continuous Discussions: Episode 25 Strategies for Deploying Mission Critical Apps at Scale

More Stories By Anders Wallgren

Anders Wallgren is Chief Technology Officer of Electric Cloud. Anders brings with him over 25 years of in-depth experience designing and building commercial software. Prior to joining Electric Cloud, Anders held executive positions at Aceva, Archistra, and Impresse. Anders also held management positions at Macromedia (MACR), Common Ground Software and Verity (VRTY), where he played critical technical leadership roles in delivering award winning technologies such as Macromedia’s Director 7 and various Shockwave products.

IoT & Smart Cities Stories
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed...
All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by ...
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next...
Cell networks have the advantage of long-range communications, reaching an estimated 90% of the world. But cell networks such as 2G, 3G and LTE consume lots of power and were designed for connecting people. They are not optimized for low- or battery-powered devices or for IoT applications with infrequently transmitted data. Cell IoT modules that support narrow-band IoT and 4G cell networks will enable cell connectivity, device management, and app enablement for low-power wide-area network IoT. B...
The hierarchical architecture that distributes "compute" within the network specially at the edge can enable new services by harnessing emerging technologies. But Edge-Compute comes at increased cost that needs to be managed and potentially augmented by creative architecture solutions as there will always a catching-up with the capacity demands. Processing power in smartphones has enhanced YoY and there is increasingly spare compute capacity that can be potentially pooled. Uber has successfully ...
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...