Click here to close now.


Linux Containers Authors: Carmen Gonzalez, Liz McMillan, Elizabeth White, AppDynamics Blog, Pat Romanski

Related Topics: Linux Containers

Linux Containers: Article

Managing an Open Source Application Stack with Aduva OnStage

A user case study

One of the most serious problems facing larger organizations considering using Linux and Open Source application stacks is managing the software environment.

The basic Linux system has adequate tools to manage a single machine, or even a small group of machines, but once the population of these machines reaches into the tens and hundreds and spreads across multiple application areas and multiple locations the management task becomes harder.

When the complete system configuration includes items not directly supported by the Linux distribution, the tools provided with the system are often not enough.

Management tasks that become non-trivial for larger installations typically are:

  • Configuration management: In larger deployments it's usual to have multiple defined configurations, each specific to a given task. Each configuration will typically have loaded only those components required for the task for which the configuration is defined, and configuration parameters in various files defined to tune the system to its specific class of task.
  • Application deployment: New application deployment, or an upgrade of an existing application, in itself not typically a difficult task, can have implications for other parts of the system if any changes to the configuration are required due to dependency requirements of the new/upgraded application. Usually not particularly difficult to manage when the application is one directly supported by the Linux distribution vendor, which will ensure that dependencies are met without disrupting the rest of the system, but potentially much more complex when it's not directly supported.
  • Loading the software on to multiple (potentially hundreds) of machines, often having to take into account factors such as performing the task when no user is present, stopping and starting system services, and possibly needing to reboot the system if any OS components are modified.
In this article, we take a look at how one large user of Linux systems handles these problems with Aduva OnStage.

Jefferson County in Colorado makes heavy use of Linux where F/OSS makes good sense. With around 60 servers to manage, the systems administrator Jeff Falgout and the team of systems engineers that he leads need all the help they can get in automating and managing configurations used for multiple purposes.

The servers run mainly Red Hat Linux 8, with a small number of SuSE servers thrown into the mix, but are slowly transitioning to Red Hat ES and AS. The application stack includes Novell e-Directory and some test installations of Fedora Directory Server (the ex-Netscape/AOL directory server closely related to Sun's JES directory server). Other services include e-mail with sendmail and SpamAssassin in conjunction with mail-scanner providing spam and virus filtering. These provide front-end filtering and routing for Novell GroupWise. Apache Web servers are used in conjunction with Tomcat application servers. Proxying/caching of user Web access is covered by Squid with appropriate anti-virus components installed. Add in a few in-house developed applications and the result is a very capable software stack, but one that if not managed correctly could give any systems administrator, as well as the IT manager, sleepless nights.

Using Aduva OnStage to manage these configurations takes a lot of the hard work out of the process and lets Jeff sleep comfortably at night.

Jeff has few worries about those components supported in the base distribution of any of the systems. With the focus that Aduva puts on the base Linux distributions, the Aduva KnowlegeBase, which OnStage depends on to ensure that all dependency requirements are met correctly, faithfully tracks the changes that distributions make as new revisions of various components are released and, especially importantly, as security issues arise. The Aduva KnowlegeBase contains all the required information to keep distributions up-to-date, avoiding known security issues or warning where avoidance might not be possible with currently available packages, while maintaining correct dependencies throughout the system.

The tracking between the Aduva KnowlegeBase and the Linux distributors' own dependency/update systems is so good that Jeff feels comfortable using them interchangeably, but in practice uses OnStage as a replacement for Linux distributors tools, such as Red Hat's to maintain a complete and accurate record of the current state of every machine. On systems that have the Red Hat applet that warns that there are system updates available from Red Hat, the applet will occasionally show red, indicating pending updates, but after the next run of OnStage, those same applets will once again show green, illustrating the close tracking between OnStage and the Linux distribution's own systems.

For stack components not directly handled in distribution base systems, Jefferson County maintains its own private extension to the KnowlegeBase, containing information on dependency requirements of those components and a repository of the RPMs containing them. All the in-house applications are managed this way, as is Tomcat, which is not part of the Red Hat base system and so not automatically included in the Aduva KnowlegeBase for Red Hat.

OnStage handles the configuration management and automatic updates of components in that configuration as they become available. Not only are system and application binaries managed, so are the various associated configuration files. OnStage can be used to build a system "from bare metal" based on any one of these configurations, or for the transformation of a system in one configuration to another. Changes are recorded and, if required, can be backed out very simply. Configurable rules can be used to schedule updates to specific machine groups and can take account of requirements such as waiting until there's no user and doing a system reboot at the end.

The ability to clone a configuration on a machine is something that Jeff uses when major updates have to be tested. The new configuration can be tested and tuned, then that identical configuration is applied to specific groups of machines once testing is complete. OnStage uses the KnowlegeBase, both the central Aduva copy and Jefferson County's own, to determine exactly which packages to remove and which to install to ensure that existing systems end up being identical to the staging/test system(s).

When asked if he trusted OnStage enough to consider skipping the verification staging build of significant changes before deployment, Jeff responded like any systems administrator faced with applying untested patches to many production system would, saying, "No, I have faith in the system, but not that much!" Then after reflection, he added, "Actually, we do allow the system updates to go straight through without testing and have never has any issues, so I guess I do trust it after all."

More Stories By Philip Peake

Philip Peake is a professional services consultant, and has worked for a variety of companies including Netscape, AOL, Sun Microsystems and OSDL.
With over 25 years experience of UNIX based systems in Internet and
Intranet enterprise environments, using Linux has been a natural evolution.
Philip has a Batchelor of Science degree in computer science from
the University of Keele in the United Kingdom.

Comments (3) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Most Recent Comments
Fraser Campbell 01/13/06 01:09:18 PM EST

Hi Philip,

I am in a position where we have 100s of SuSE servers and that number is expected to grow by leaps and bounds as virtualization begins to take hold.

I agree that higher level management tools are critical to managing this infrastructure.

I am curious as to what tools you may have used yourself, have you for example considered cfengine. My bias is weighted heavily towards open source tools despite the fact that we are using SLES servers.

news desk 12/27/05 11:43:26 PM EST

One of the most serious problems facing larger organizations considering using Linux and Open Source application stacks is managing the software environment. The basic Linux system has adequate tools to manage a single machine, or even a small group of machines, but once the population of these machines reaches into the tens and hundreds and spreads across multiple application areas and multiple locations the management task becomes harder.

news desk 12/27/05 11:32:19 PM EST

One of the most serious problems facing larger organizations considering using Linux and Open Source application stacks is managing the software environment. The basic Linux system has adequate tools to manage a single machine, or even a small group of machines, but once the population of these machines reaches into the tens and hundreds and spreads across multiple application areas and multiple locations the management task becomes harder.

@ThingsExpo Stories
SYS-CON Events announced today that Dyn, the worldwide leader in Internet Performance, will exhibit at SYS-CON's 17th International Cloud Expo®, which will take place on November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Dyn is a cloud-based Internet Performance company. Dyn helps companies monitor, control, and optimize online infrastructure for an exceptional end-user experience. Through a world-class network and unrivaled, objective intelligence into Internet conditions, Dyn ensures traffic gets delivered faster, safer, and more reliably than ever.
Today air travel is a minefield of delays, hassles and customer disappointment. Airlines struggle to revitalize the experience. GE and M2Mi will demonstrate practical examples of how IoT solutions are helping airlines bring back personalization, reduce trip time and improve reliability. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Dr. Sarah Cooper, M2Mi's VP Business Development and Engineering, will explore the IoT cloud-based platform technologies driving this change including privacy controls, data transparency and integration of real time context w...
Who are you? How do you introduce yourself? Do you use a name, or do you greet a friend by the last four digits of his social security number? Assuming you don’t, why are we content to associate our identity with 10 random digits assigned by our phone company? Identity is an issue that affects everyone, but as individuals we don’t spend a lot of time thinking about it. In his session at @ThingsExpo, Ben Klang, Founder & President of Mojo Lingo, will discuss the impact of technology on identity. Should we federate, or not? How should identity be secured? Who owns the identity? How is identity ...
The IoT market is on track to hit $7.1 trillion in 2020. The reality is that only a handful of companies are ready for this massive demand. There are a lot of barriers, paint points, traps, and hidden roadblocks. How can we deal with these issues and challenges? The paradigm has changed. Old-style ad-hoc trial-and-error ways will certainly lead you to the dead end. What is mandatory is an overarching and adaptive approach to effectively handle the rapid changes and exponential growth.
The buzz continues for cloud, data analytics and the Internet of Things (IoT) and their collective impact across all industries. But a new conversation is emerging - how do companies use industry disruption and technology enablers to lead in markets undergoing change, uncertainty and ambiguity? Organizations of all sizes need to evolve and transform, often under massive pressure, as industry lines blur and merge and traditional business models are assaulted and turned upside down. In this new data-driven world, marketplaces reign supreme while interoperability, APIs and applications deliver un...
Too often with compelling new technologies market participants become overly enamored with that attractiveness of the technology and neglect underlying business drivers. This tendency, what some call the “newest shiny object syndrome,” is understandable given that virtually all of us are heavily engaged in technology. But it is also mistaken. Without concrete business cases driving its deployment, IoT, like many other technologies before it, will fade into obscurity.
Electric power utilities face relentless pressure on their financial performance, and reducing distribution grid losses is one of the last untapped opportunities to meet their business goals. Combining IoT-enabled sensors and cloud-based data analytics, utilities now are able to find, quantify and reduce losses faster – and with a smaller IT footprint. Solutions exist using Internet-enabled sensors deployed temporarily at strategic locations within the distribution grid to measure actual line loads.
The Internet of Everything is re-shaping technology trends–moving away from “request/response” architecture to an “always-on” Streaming Web where data is in constant motion and secure, reliable communication is an absolute necessity. As more and more THINGS go online, the challenges that developers will need to address will only increase exponentially. In his session at @ThingsExpo, Todd Greene, Founder & CEO of PubNub, will explore the current state of IoT connectivity and review key trends and technology requirements that will drive the Internet of Things from hype to reality.
The Internet of Things (IoT) is growing rapidly by extending current technologies, products and networks. By 2020, Cisco estimates there will be 50 billion connected devices. Gartner has forecast revenues of over $300 billion, just to IoT suppliers. Now is the time to figure out how you’ll make money – not just create innovative products. With hundreds of new products and companies jumping into the IoT fray every month, there’s no shortage of innovation. Despite this, McKinsey/VisionMobile data shows "less than 10 percent of IoT developers are making enough to support a reasonably sized team....
You have your devices and your data, but what about the rest of your Internet of Things story? Two popular classes of technologies that nicely handle the Big Data analytics for Internet of Things are Apache Hadoop and NoSQL. Hadoop is designed for parallelizing analytical work across many servers and is ideal for the massive data volumes you create with IoT devices. NoSQL databases such as Apache HBase are ideal for storing and retrieving IoT data as “time series data.”
Today’s connected world is moving from devices towards things, what this means is that by using increasingly low cost sensors embedded in devices we can create many new use cases. These span across use cases in cities, vehicles, home, offices, factories, retail environments, worksites, health, logistics, and health. These use cases rely on ubiquitous connectivity and generate massive amounts of data at scale. These technologies enable new business opportunities, ways to optimize and automate, along with new ways to engage with users.
The IoT is upon us, but today’s databases, built on 30-year-old math, require multiple platforms to create a single solution. Data demands of the IoT require Big Data systems that can handle ingest, transactions and analytics concurrently adapting to varied situations as they occur, with speed at scale. In his session at @ThingsExpo, Chad Jones, chief strategy officer at Deep Information Sciences, will look differently at IoT data so enterprises can fully leverage their IoT potential. He’ll share tips on how to speed up business initiatives, harness Big Data and remain one step ahead by apply...
There will be 20 billion IoT devices connected to the Internet soon. What if we could control these devices with our voice, mind, or gestures? What if we could teach these devices how to talk to each other? What if these devices could learn how to interact with us (and each other) to make our lives better? What if Jarvis was real? How can I gain these super powers? In his session at 17th Cloud Expo, Chris Matthieu, co-founder and CTO of Octoblu, will show you!
As a company adopts a DevOps approach to software development, what are key things that both the Dev and Ops side of the business must keep in mind to ensure effective continuous delivery? In his session at DevOps Summit, Mark Hydar, Head of DevOps, Ericsson TV Platforms, will share best practices and provide helpful tips for Ops teams to adopt an open line of communication with the development side of the house to ensure success between the two sides.
SYS-CON Events announced today that ProfitBricks, the provider of painless cloud infrastructure, will exhibit at SYS-CON's 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. ProfitBricks is the IaaS provider that offers a painless cloud experience for all IT users, with no learning curve. ProfitBricks boasts flexible cloud servers and networking, an integrated Data Center Designer tool for visual control over the cloud and the best price/performance value available. ProfitBricks was named one of the coolest Clo...
SYS-CON Events announced today that IBM Cloud Data Services has been named “Bronze Sponsor” of SYS-CON's 17th Cloud Expo, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. IBM Cloud Data Services offers a portfolio of integrated, best-of-breed cloud data services for developers focused on mobile computing and analytics use cases.
SYS-CON Events announced today that Sandy Carter, IBM General Manager Cloud Ecosystem and Developers, and a Social Business Evangelist, will keynote at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
Developing software for the Internet of Things (IoT) comes with its own set of challenges. Security, privacy, and unified standards are a few key issues. In addition, each IoT product is comprised of at least three separate application components: the software embedded in the device, the backend big-data service, and the mobile application for the end user's controls. Each component is developed by a different team, using different technologies and practices, and deployed to a different stack/target - this makes the integration of these separate pipelines and the coordination of software upd...
Mobile messaging has been a popular communication channel for more than 20 years. Finnish engineer Matti Makkonen invented the idea for SMS (Short Message Service) in 1984, making his vision a reality on December 3, 1992 by sending the first message ("Happy Christmas") from a PC to a cell phone. Since then, the technology has evolved immensely, from both a technology standpoint, and in our everyday uses for it. Originally used for person-to-person (P2P) communication, i.e., Sally sends a text message to Betty – mobile messaging now offers tremendous value to businesses for customer and empl...
"Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.