|By Pete Pickerill||
|March 26, 2014 12:00 PM EDT||
In the final post in this series about bringing DevOps patterns to database change management, we’re going to discuss the Third Way. Here’s a refresher on the Third Way from the introductory post in this series:
The Third Way: Culture of Continual Experimentation & Learning – This way emphasizes the benefits that can be realized through embracing experimentation, risk-taking, and learning from failure. By adopting this kind of attitude, experimentation and risk-taking lead to innovation and improvement while embracing failure allows the organization to produce more resilient products and sharpen skills that allow teams to recover more quickly from unexpected failure when it does occur.
The Third Way is by far the most intriguing of the “The Ways” to me. I’ve spent the lion’s share of my career in early stage start-ups where cycles of experimentation, learning, and failure are the norm. When bringing a new product or service to market, your latest release is never your last. It may not even be the last release this week. You are constantly experimenting with new workflows and technology, learning about your target market, and getting more valuable information from your early failures than your early successes. The Third Way is crucial to the success of an early stage company.
While the benefits of the Third Way still apply to more established companies and product lines, practicing it becomes more difficult. The potential negatives of experimentation and risk-taking are much harder to stomach when you have a large base of paying customers with SLAs. This aversion to risk is most acute when you’re talking about your data platform where outages, performance problems and data loss are not an option. Complicating matters further is how difficult it can be to unwind the database changes that were affected to support a specific version of your app. Application code can usually be uninstalled and replaced with the previous working version fairly simply should problems arise. Reverting the database changes that support that version of the application is more akin to defusing a bomb. Database changes must be reverted delicately and meticulously to avoid errors and omissions that could negatively impact your data platform.
What DBAs and release managers need to facilitate experimentation and risk taking on the data platform is a special combination of tools and process. This combination should make it easy to identify the root cause of issues, quickly remediate problems caused by application schema structure, and revert to a previous version of the schema safely. When we started Datical, we spent several hours in conversation and at white boards exploring these unique needs and hammering out a path to usher the Third Way into regular database activity.
A Rollback Designed With Every Change
The biggest problem with experimentation in the data platform is how difficult it is to move backward and forward through your schema’s version history. We feel the best way to bring more flexibility to the process of upgrading and reverting schema is an attitude shift. Your rollback strategy for each database change must become as important as the change itself. The best time to craft your rollback strategy is when the change itself is being designed. When the motivation for the change is fresh in your mind and the dependencies of the object being created, dropped or altered are clearly mapped out, a developer or DBA can better craft a rollback strategy for which every contingency has been considered. This leads to a stronger safety net and makes your application schema as agile and easily managed as your application code. The database is no longer preventing you from being bold but quickly and safely moving between versions to accommodate experimentation.
The Richness of Model Based Comparison & Remediation
I’m a native Texan, born and raised in Austin. I know the jokes about how proud and vocal Texans can be about their home state. That being said, I spend more time telling people about the model based approach Datical has applied to Database Change Management than I do telling them how wonderful it is that I was fortunate enough to be born in the best state in the country. The advantages of the model based approach really shine through when it comes to experimentation and troubleshooting. The model allows you to annotate all objects and modifications to your application’s schema with the business reason that prompted them. This detailed history is invaluable when designing new changes or refactoring your schema as part of an experimental exercise. You immediately know what objects are most crucial, what your dependencies are, what areas you need to tread lightly in, and what areas are ripe for experimentation due to the changing needs of your business. Designing intelligently eliminates risk.
Troubleshooting with models is also dramatically faster and more reliable than other methods. Programmatically comparing models allows you to determine the differences between two databases much more quickly than manually comparing diagrams or SQL scripts. You know with certainty exactly what has changed and what is missing in a fraction of the time that human review takes.
Once you have identified the differences, remediation is as simple plugging one, some or all of the determined differences into the model and deploying those changes to the non-compliant instance.
Flexible Quick Rollback
If you’ve taken our advice to implement your rollback strategy when you implement a change, recovering from disaster becomes testable, fast, and simple. Before going to production or a sensitive environment, you should always test your rollback steps in dev, test and staging. This will allow you to make any tweaks or changes to your rollback strategy before you are in a pinch. Think of it like testing your smoke alarm. Hopefully you’ll never need it, but it’s nice to know that it’ll work if you do.
Let’s say the worst happens. You deploy a new set of database changes and the application performance degrades or errors are logged. The decision is made to revert the entire installation to the previous version. Because you have carefully designed, tested and refined your rollback strategy, rolling back the database changes becomes a push button operation or a single invocation of a command line tool. No more running disparate SQL scripts or undoing changes on the fly. You can be confident that your database has been returned to the same state it was in before the upgrade.
The database has long been handled with kid gloves and for good reason. Data is a precious resource for consumers and businesses. Consumers provide data to businesses and trust that it will be kept safe and used to offer them better products and services. Businesses rely on data to strategize, grow, and become more efficient and profitable. It is the lifeblood of our economy. As our ability to collect and process data becomes greater and greater, the rate at which an enterprise must move on what is learned from that data becomes linearly faster. Data must be kept safe, but the database must become more agile to accommodate the growing pressure for faster value realization of business intelligence initiatives. DevOps patterns hold the key to this necessary agility while maintaining or improving the security and integrity of data stores. The database and DBAs need to be brought into the application design process earlier and must be treated as the first class stakeholder and commodity that they are. Companies that acknowledge this and move to adopt DevOps patterns and include their database teams will be at a distinct competitive advantage to those that don’t. Don’t miss the boat!
Microservices are a very exciting architectural approach that many organizations are looking to as a way to accelerate innovation. Microservices promise to allow teams to move away from monolithic "ball of mud" systems, but the reality is that, in the vast majority of organizations, different projects and technologies will continue to be developed at different speeds. How to handle the dependencies between these disparate systems with different iteration cycles? Consider the "canoncial problem" in this scenario: microservice A (releases daily) depends on a couple of additions to backend B (re...
Dec. 1, 2015 09:00 AM EST Reads: 481
Too often with compelling new technologies market participants become overly enamored with that attractiveness of the technology and neglect underlying business drivers. This tendency, what some call the “newest shiny object syndrome” is understandable given that virtually all of us are heavily engaged in technology. But it is also mistaken. Without concrete business cases driving its deployment, IoT, like many other technologies before it, will fade into obscurity.
Dec. 1, 2015 08:00 AM EST Reads: 395
We all know that data growth is exploding and storage budgets are shrinking. Instead of showing you charts on about how much data there is, in his General Session at 17th Cloud Expo, Scott Cleland, Senior Director of Product Marketing at HGST, showed how to capture all of your data in one place. After you have your data under control, you can then analyze it in one place, saving time and resources.
Dec. 1, 2015 08:00 AM EST Reads: 251
Container technology is shaping the future of DevOps and it’s also changing the way organizations think about application development. With the rise of mobile applications in the enterprise, businesses are abandoning year-long development cycles and embracing technologies that enable rapid development and continuous deployment of apps. In his session at DevOps Summit, Kurt Collins, Developer Evangelist at Built.io, examined how Docker has evolved into a highly effective tool for application delivery by allowing increasingly popular Mobile Backend-as-a-Service (mBaaS) platforms to quickly crea...
Dec. 1, 2015 08:00 AM EST Reads: 396
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound effect on the world, and what should we expect to see over the next couple of years.
Dec. 1, 2015 06:30 AM EST Reads: 515
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Day 2 Keynote at 17th Cloud Expo, Sandy Carter, IBM General Manager Cloud Ecosystem and Developers, and a Social Business Evangelist, wil...
Dec. 1, 2015 05:00 AM EST Reads: 620
PubNub has announced the release of BLOCKS, a set of customizable microservices that give developers a simple way to add code and deploy features for realtime apps.PubNub BLOCKS executes business logic directly on the data streaming through PubNub’s network without splitting it off to an intermediary server controlled by the customer. This revolutionary approach streamlines app development, reduces endpoint-to-endpoint latency, and allows apps to better leverage the enormous scalability of PubNub’s Data Stream Network.
Dec. 1, 2015 05:00 AM EST Reads: 359
Apps and devices shouldn't stop working when there's limited or no network connectivity. Learn how to bring data stored in a cloud database to the edge of the network (and back again) whenever an Internet connection is available. In his session at 17th Cloud Expo, Ben Perlmutter, a Sales Engineer with IBM Cloudant, demonstrated techniques for replicating cloud databases with devices in order to build offline-first mobile or Internet of Things (IoT) apps that can provide a better, faster user experience, both offline and online. The focus of this talk was on IBM Cloudant, Apache CouchDB, and ...
Dec. 1, 2015 04:45 AM EST Reads: 458
I recently attended and was a speaker at the 4th International Internet of @ThingsExpo at the Santa Clara Convention Center. I also had the opportunity to attend this event last year and I wrote a blog from that show talking about how the “Enterprise Impact of IoT” was a key theme of last year’s show. I was curious to see if the same theme would still resonate 365 days later and what, if any, changes I would see in the content presented.
Dec. 1, 2015 03:00 AM EST Reads: 470
Cloud computing delivers on-demand resources that provide businesses with flexibility and cost-savings. The challenge in moving workloads to the cloud has been the cost and complexity of ensuring the initial and ongoing security and regulatory (PCI, HIPAA, FFIEC) compliance across private and public clouds. Manual security compliance is slow, prone to human error, and represents over 50% of the cost of managing cloud applications. Determining how to automate cloud security compliance is critical to maintaining positive ROI. Raxak Protect is an automated security compliance SaaS platform and ma...
Dec. 1, 2015 03:00 AM EST Reads: 469
Most of the IoT Gateway scenarios involve collecting data from machines/processing and pushing data upstream to cloud for further analytics. The gateway hardware varies from Raspberry Pi to Industrial PCs. The document states the process of allowing deploying polyglot data pipelining software with the clear notion of supporting immutability. In his session at @ThingsExpo, Shashank Jain, a development architect for SAP Labs, discussed the objective, which is to automate the IoT deployment process from development to production scenarios using Docker containers.
Dec. 1, 2015 01:15 AM EST Reads: 127
Countless business models have spawned from the IaaS industry – resell Web hosting, blogs, public cloud, and on and on. With the overwhelming amount of tools available to us, it's sometimes easy to overlook that many of them are just new skins of resources we've had for a long time. In his general session at 17th Cloud Expo, Harold Hannon, Sr. Software Architect at SoftLayer, an IBM Company, broke down what we have to work with, discussed the benefits and pitfalls and how we can best use them to design hosted applications.
Nov. 30, 2015 03:45 PM EST Reads: 116
The Internet of Things (IoT) is growing rapidly by extending current technologies, products and networks. By 2020, Cisco estimates there will be 50 billion connected devices. Gartner has forecast revenues of over $300 billion, just to IoT suppliers. Now is the time to figure out how you’ll make money – not just create innovative products. With hundreds of new products and companies jumping into the IoT fray every month, there’s no shortage of innovation. Despite this, McKinsey/VisionMobile data shows "less than 10 percent of IoT developers are making enough to support a reasonably sized team....
Nov. 30, 2015 03:00 PM EST Reads: 497
Just over a week ago I received a long and loud sustained applause for a presentation I delivered at this year’s Cloud Expo in Santa Clara. I was extremely pleased with the turnout and had some very good conversations with many of the attendees. Over the next few days I had many more meaningful conversations and was not only happy with the results but also learned a few new things. Here is everything I learned in those three days distilled into three short points.
Nov. 30, 2015 02:00 PM EST Reads: 375
DevOps is about increasing efficiency, but nothing is more inefficient than building the same application twice. However, this is a routine occurrence with enterprise applications that need both a rich desktop web interface and strong mobile support. With recent technological advances from Isomorphic Software and others, rich desktop and tuned mobile experiences can now be created with a single codebase – without compromising functionality, performance or usability. In his session at DevOps Summit, Charles Kendrick, CTO and Chief Architect at Isomorphic Software, demonstrated examples of com...
Nov. 30, 2015 01:45 PM EST Reads: 439
As organizations realize the scope of the Internet of Things, gaining key insights from Big Data, through the use of advanced analytics, becomes crucial. However, IoT also creates the need for petabyte scale storage of data from millions of devices. A new type of Storage is required which seamlessly integrates robust data analytics with massive scale. These storage systems will act as “smart systems” provide in-place analytics that speed discovery and enable businesses to quickly derive meaningful and actionable insights. In his session at @ThingsExpo, Paul Turner, Chief Marketing Officer at...
Nov. 30, 2015 01:45 PM EST Reads: 442
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
Nov. 30, 2015 01:00 PM EST Reads: 542
In his General Session at 17th Cloud Expo, Bruce Swann, Senior Product Marketing Manager for Adobe Campaign, explored the key ingredients of cross-channel marketing in a digital world. Learn how the Adobe Marketing Cloud can help marketers embrace opportunities for personalized, relevant and real-time customer engagement across offline (direct mail, point of sale, call center) and digital (email, website, SMS, mobile apps, social networks, connected objects).
Nov. 30, 2015 12:45 PM EST Reads: 347
The Internet of Everything is re-shaping technology trends–moving away from “request/response” architecture to an “always-on” Streaming Web where data is in constant motion and secure, reliable communication is an absolute necessity. As more and more THINGS go online, the challenges that developers will need to address will only increase exponentially. In his session at @ThingsExpo, Todd Greene, Founder & CEO of PubNub, exploreed the current state of IoT connectivity and review key trends and technology requirements that will drive the Internet of Things from hype to reality.
Nov. 30, 2015 10:45 AM EST Reads: 469
Two weeks ago (November 3-5), I attended the Cloud Expo Silicon Valley as a speaker, where I presented on the security and privacy due diligence requirements for cloud solutions. Cloud security is a topical issue for every CIO, CISO, and technology buyer. Decision-makers are always looking for insights on how to mitigate the security risks of implementing and using cloud solutions. Based on the presentation topics covered at the conference, as well as the general discussions heard between sessions, I wanted to share some of my observations on emerging trends. As cyber security serves as a fou...
Nov. 30, 2015 10:30 AM EST Reads: 361