From the Blogosphere

Hadoop Quickstart: Create and Better Manage Hadoop Clusters on Rackspace

Fri, 17 Feb 2012 15:00:00 EST

We have previously provided a Quickstart guide to standing up Rackspace cloud servers (and have one for Amazon servers as well). These are very low cost ways of building reliable, production ready capabilities for enterprise use (commercial and government).

Mid-Market = Semi-Big Data?

Thu, 16 Feb 2012 06:15:00 EST

Is Big Data destined for only the top 3,000 companies worldwide? What about medium or small companies who are equally as data-driven? Is there a place for Big Data in SMB markets? When I talk to SMB companies about their use of public cloud services, it’s a no-brainer. Pay as you go, lower costs upfront, quick time-to-market. With private and public cloud solutions, both big and small companies benefit.

Startup Debuts with Encryption for Cloud Computing

Thu, 16 Feb 2012 02:07:00 EST

Israel-based startup Porticor launches this week with technology aimed at giving enterprises a way to encrypt data held in cloud computing services, including those from Amazon and Rackspace. Porticor Virtual Private Data is focused on protecting data at rest in cloud-based computing centers where customers rent disk space or servers, says Gilad Parann-Nissany, Porticor’s co-founder and CEO. What Porticor is doing that’s distinct is relying on a so-called “split-key” method in which the service to encrypt and decrypt doesn’t work unless both pieces of the key are together. The enterprise customer holds what Porticor calls the “master key,” and the idea is to foster trust by putting the customer in complete control. “We don’t have the customer master key,” says Parann-Nissany. In fact, the service provider doesn’t even see the master key in the encryption method that’s applied.

New iOS Edge Client

Wed, 15 Feb 2012 15:28:00 EST

If you are running the BIG-IP Edge Client on your iPhone, iPod or iPad, you may have gotten an AppStore alert for an update. If not, I just wanted to let you know that version 1.0.3 of the iOS Edge Client is available at the AppStore. The main updates in v1.0.3: URI scheme enhancement allows passing configuration data to the client upon access. For example, you could have a link on the WebTop that invokes the client and forces web logon mode. Other Bug fixes.

Truth in (Round) Numbers?

Wed, 15 Feb 2012 10:30:00 EST

Statistics matter, not only in business, but increasingly also in our social life - well, at least in our social media life. Some of the statistics I noticed this week were round numbers, like 1000. With 1000 representing both the number now showing under "followers" in Twitter and the revenue number for research (that's excluding events, consulting and other items) we grew to in 2011. And on my blog I saw - a bit to my surprise - it has been a full 10 weeks since my last post! That's however more a case of blogger's block than writer's block, as I did (co-)author the round number of 10 research notes since joining this summer. To catch up, I am including below a short overview of the topics these research notes covered (Gartner clients only) and that I likely will explore further in the future - both in research and using (social) media. So what topics did these 10 research notes address? First to mention are the Predicts 2012. I participated in two this year, one called Predicts 2012: Cloud Computing Is Becoming a Reality in which we revisited an earlier prediction on cloud lock-in and explored the idea of a Maslov type hierarchy of needs for cloud computing customers. In this needs hierarchy fear of lock-in will be gaining ground as more basic needs like security are better understood.

Oops! HTML5 Does It Again

Wed, 15 Feb 2012 07:28:00 EST

#HTML5 #infosec A multitude of security-related solutions rely upon the ability to extract and examine mime-objects from web-content. HTML5 may significantly impair their ability to do so.

The trade off between security and performance has long been a known issue across IT organizations. One of the first things to go when performance is unacceptable is a security solution. This isn’t just an IT phenomenon either; consider how many of us have disabled endpoint security solutions like anti-virus scanners to improve performance?

Our refusal to be slowed down by what may seem to some as extraneous security is what eventually led IT security professionals to revise their strategies and enforce such scans on inbound content in the network. Network-attached security scanning solutions have long been a staple of inbound e-mail and has found increasing use as a means to scan inbound web-content, as well, as an attempt to eliminate potential malware from having access to the corporate network.

IT Organizations That Trade Security for Performance Deserve Neither

A new [at the time of publication, July 2011] survey of 487 IT professionals that was conducted by Crossbeam, a provider of high-performance security gateways, finds that while 91 percent of the respondents were not only making tradeoffs between security and performance, a full 81 percent were actually disabling security features.

HTML and soon, if we believe the predictions HTML5, is the lingua franca of Internet communication. Oh, applications may speak JSON under the covers, but in the end it’s just data to be displayed to the user which means HTML(5).

What does that mean for anti-virus and malware web scanners? Well, if one of the features of HTML5 being leveraged is WebSockets, a lot. Otherwise, not much. At least not yet.

You see, WebSockets accidentally trades performance for security.

OOPS

One of the things WebSockets does to dramatically improve performance is eliminate all those pesky HTTP headers. You know, things like CONTENT-TYPE. You know, the header that tells the endpoint what kind of content is being transferred, such as text/html and video/avi. One of the things anti-virus and malware scanning solutions are very good at is detecting anomalies in specific types of content. The problem is that without a MIME type, the ability to correctly identify a given object gets a bit iffy. Bits and bytes are bytes and bytes, and while you could certainly infer the type based on format “tells” within the actual data, how would you really know? Sure, the HTTP headers could by lying, but generally speaking the application serving the object doesn’t lie about the type of data and it is a rare vulnerability that attempts to manipulate that value. After all, you want a malicious payload delivered via a specific medium, because that’s the cornerstone upon which many exploits are based – execution of a specific operation against a specific manipulated payload. That means you really need the endpoint to believe the content is of the type it thinks it is.

But couldn’t you just use the URL? Nope – there is no URL associated with objects via a WebSocket. There is also no standard application information that next-generation firewalls can use to differentiate the content; developers are free to innovate and create their own formats and micro-formats, and undoubtedly will. And trying to prevent its use is nigh-unto impossible because of the way in which the upgrade handshake is performed – it’s all over HTTP, and stays HTTP. One minute the session is talking understandable HTTP, the next they’re whispering in Lakota, a traditionally oral-only language which neatly illustrates the overarching point of this post thus far: there’s no way to confidently know what is being passed over a WebSocket unless you “speak” the language used, which you may or may not have access to.

The result of all this confusion is that security software designed to scan for specific signatures or anomalies within specific types of content can’t. They can’t extract the object flowing through a WebSocket because there’s no indication of where it begins or ends, or even what it is. The loss of HTTP headers that indicate not only type but length is problematic for any software – or hardware for that matter – that uses the information contained within to extract and process the data.

WEDGE NETWORKS

Wedge Networks, whose name you may never before heard even though you might have had content scrubbed by their devices and not known it, has a solution to the problem of disaggregating web objects without requiring specific identification by HTTP headers, thus solving this problem and several other similar ones where protocols lack the means to definitively identify specific content by type.

WedgeOS - Network Data Processor Architecture

The WedgeOS Network Data Processor ("NDP") is the proprietary architecture that allows content inspection at Gigabit speeds without impacting network performance. The WedgeOS NDP architecture revolutionized Web Security Appliances with the introduction of BeSecure. BeSecure is capable of intercepting and actively scanning all internet traffic for malicious content as it enters the network.

What they meant to say was “we do deep content inspection on streaming traffic and are able to accurately identify – and subsequently extract – MIME objects at line rate and then scan them for bad stuff you don’t want on your network.” Content comes into their device (and it’s off-the shelf hardware, I’m told), MIME objects are disaggregated regardless of transport or application protocol, shoved down a high-speed internal bus into which are plugged a variety of security scanning functions, and then shoved back out the other side, assuming all was well. Policies enable the ability to determine exactly what happens if there are anomalies or malicious code discovered.

Wedge Networks has partnered with a number of well-known and industry leading security scanning solutions and brought them together into a single device. Applying the old “crack the packet only once” doctrine, the device is able to perform its scans as fast as objects can traverse its internal bus.

The devices deploys in either proxy or transparent mode, with the latter being most popular simply due to the mitigation of disruption that can come with inserting a proxy-based solution into an established network.

Let’s assume for a moment that a Wedge Networks device really does accomplish all this – at line rate. I can’t know, I don’t evaluate products in lab environments any more, so I can take their word for it. But let’s assume it does. That opens a wide variety of possibilities – both inbound and outbound – for protecting web applications and customers alike, and not just for HTML5.

Assuming no degradation of overall performance, the ability to detect and prevent delivery of malware that may have been surgically inserted into your database or CMS via XSS or SQLi would be a boon, if only to let you know it happened much sooner and provide the time necessary to redress the infection. Nearly every rational organization scans inbound e-mail for potential risks, but very few (if any) scan outbound. We all know why – the belief that performance is more important than security, especially when consumer dollars are on the line. If Wedge Networks can do as it promises and not impede performance while still providing a valuable security service, well, that might be something to think about.

Connect with Lori:	Connect with F5:

Related blogs & articles:

Technorati Tags: F5,MacVittie,Wedge Networks,network,security,HTML5,WebSockets,malware,anti-virus,performance,application security,blog

The Path to the Intelligent Cloud

Wed, 15 Feb 2012 07:00:00 EST

Let's face it right now the cloud is pretty immature. The level of automation and management of these environments are analogous to the early assembly lines, but it won't be this way long. This is not the industrial revolution and it moves at a wicked fast pace. Before we know it the next generation of cloud computing will be upon us and it will be very different than the IaaS/PaaS/SaaS offerings we know today. For one, it will be intelligent. That is, the cloud will be content aware and it's network connections will act like mycelia hyphae and what one hyphae learns will become available to the entire cloud. Whereas the current cloud is focused on scalability and elasticity, the next instance of the cloud will focus on redundancy, resiliency and collaboration. The discussion regarding public, private or hybrid will become moot as the cloud simply becomes a system of nodes with some nodes participating fully while some don't participate at all. Nodes will contribute to the cloud on a controlled basis. Some will host their own nodes while others will pay service providers to host their nodes for them.

How Hot Is Cloud Computing?

Wed, 15 Feb 2012 04:32:00 EST

In previous posts such as Cloud Computing: Hype, Vision or Reality?, Hyped Cloud Technologies, PAAS is not Mainstream yet, SaaS is going Mainstream, Future applications: SaaS or traditional? I discussed Cloud Computing. Recently I read Joe McKendrick's interesting article titled:Cloud Computing Market Hot, But How Hot? Estimates are All Over the Map. Joe's views and predictions are similar to mine. However, he based his opinions on actual survey done by him, as part of his work with Unisphere Research/Information Today Inc and on leading Analyst firms predictions.

Analyzing the PaaS Landscape – Microsoft Windows Azure

Tue, 14 Feb 2012 21:29:00 EST

Having covered Cloud Foundry, Force.com, Google App Engine and Red Hat OpenShift, we now take a look at Microsoft’s PaaS offering, Windows Azure. Microsoft Windows Azure Platform is a Platform as a Service offering from Microsoft. It was announced in 2008 and became available in 2010. Since then Microsoft has been constantly improving the platform by adding new features

Virtualization Certification(s) - Is It Really Worth It?

Tue, 14 Feb 2012 12:43:21 EST

Many virtualization vendors offer certifications. With that in mind, is there really any value in pursuing these certifications from Microsoft and VMware? Is one more "valuable" than the other? First, let me say that I am a big proponent of technical certifications. That is the reason why I have my Microsoft Certified IT Professional (MCITP) Server Administrator, MCITP Enterprise Administrator and MCITP Enterprise Messaging Administrator 2007 and 2010. Once upon a time, I also had my Microsoft Certified Trainer (MCT), MCSE (precursor to MCITP), CNI and CNE. Recently, I passed Windows Server 2008 R2 Server Virtualization (Exam 70-659) and Microsoft System Center Operations Manager 2007 (Exam 70-400). I plan to get my MCITP Virtualization Administrator so I'll need to take the Windows Server 2008 R2 Desktop Virtualization Exam (70-669) and Windows Server 2008 R2 Virtualization Administrator Exam (70-693) sometime in the next few months. As soon as the two exams for Microsoft Private Cloud Certification become available (Exams 70-246 and 70-247), I will also study and take those.

What Does Mobile Mean, Anyway?

Tue, 14 Feb 2012 10:30:00 EST

There are – according to about a bazillion studies - 4 billion mobile devices in use around the globe. It is interesting to note that nearly everyone who notes this statistic and then attempts to break it down into useful data (usually for marketing) that they almost always do so based on OS or device type – but never, ever, ever based on connectivity. Consider the breakdown offered by W3C for October 2011. Device type is the chosen taxonomy, with operating system being the alternative view. Unfortunately, aside from providing useful trending on device type for application developers and organizations, this data does not provide the full range of information necessary to actually make these devices, well, useful.

Measuring Cloud Storage Performance: Blocks vs. Files

Tue, 14 Feb 2012 10:00:00 EST

What are some good reasons to adopt cloud storage? Cost, durability and flexibility. So let me talk about performance, instead. As part of our daily testing, we do routine performance measurements across a broad swath of cloud storage providers. It gives us a check to ensure that the various CloudArray subsystems are performing as they should, and gives us the data to make optimization decisions. In this particular test, we measure transfer rates at various buffer sizes. We “fill the pipe” by queueing up multiple streams of data simultaneously, initiating one transfer as soon as the previous one finishes, so that latency doesn’t skew the data.

Bare Metal Restore in the Cloud: A How-to-Guide

Tue, 14 Feb 2012 08:15:00 EST

They all automatically combine disaster recovery with backup, since the backups are stored offsite at the cloud provider’s data center. The better cloud backup options completely automate both backup and restore, removing what historically has been a complex, order-and process-intensive, manual task. The very best-in-class solutions will automatically create rich version histories, and also provide you with a secured, online replica copy of your data, that you can access from anywhere using just an Internet browser. However, these solutions, no matter how robust, cannot circumvent the laws of physics. The reality is that you will be using your Internet connection as a conduit to move large data sets. This prescribes slightly modified best practices, including reconsidering the benefits of bare metal restores.

OAuth Token Management

Mon, 13 Feb 2012 14:00:00 EST

Tokens are at the center of API access control in the Enterprise. Token management, the process through which the lifecycle of these tokens is governed emerges as an important aspect of Enterprise API Management. While some of this information is created during OAuth handshakes, some of it continues to evolve throughout the lifespan of the token. Token management is used during handshakes to capture all relevant information pertaining to granting access to an API and makes this information available to other relevant API management components at runtime.

Getting Ready for Big Data

Mon, 13 Feb 2012 08:00:00 EST

IT departments and data centers are used to seeing demand for resources surge. In recent years, this has been especially evident in the area of data storage. No matter what you want to call it – “data explosion,” or something else – you can’t deny the fact that organizations simply have a greater need for capacity today than ever before. Add in emerging technologies such as real-time predictive intelligent solutions, and you can begin to see how the demand for data is going to continue to explode. Fortunately, technology is moving to keep pace. There is a technology coming down the pipe known as “big data,” and it’s going to change the way that IT and data centers handle all of that excess demand.

Enterprise Cloud Services – SAP

Mon, 13 Feb 2012 05:00:00 EST

As the name suggests one of the key factors of ‘Enterprise Cloud’ is that it’s intended for the enterprise market, in particular the enterprise applications that they use such as SAP, Oracle and JD Edwards amongst others. Where Cloud Computing overlaps with this sector is ‘Enterprise Cloud Computing‘, and you can follow this topic via … Continue reading »

Yellow Pencil – Web Channel as a Service

Sun, 12 Feb 2012 15:36:00 EST

Last week we ran our ‘MaaS’ webinar – Municipality as a Service, and we’re now finalizing all the individual presentations to be available via the follow on newsletter that’s being launched : MunicipalCloud.biz. One of these presentations is from Paul Bellows of Yellow Pencil: 6-page PDF Specializing in web 2.0 content management systems like OpenText and Drupal, Paul led us through examples of how Yellow Pencil has helped Municipalities like Edmonton to use these platforms to run multiple internal and external web sites for citizen engagement, leverage Open Data and many other aspects of harnessing the technology to better manage their ‘Web Channel as a Service’. In this day and age the public sector like Municipalities are required to use more social media and in a variety of new ways too, a mix of new IT and business processes that are new and overwhelming for many agencies. Yellow provides this expertise ‘as a Service’ as much as they do the underlying software that implements these practices to help with this adoption cycle.

Thoughts on the "Apps Economy," "Open Data" and the Value of APIs

Sat, 11 Feb 2012 18:22:00 EST

To quote my friend Stevie Chambers (@stevie_chambers), "I feel like a new room has opened in my memory palace." That was exactly how I felt after finishing my recent The Cloudcast (.net) podcast with Sam Ramji (@sramji) and Christian Reilly (@reillyusa), where we discussed the role of APIs in the economy of mobile devices and "open data". I had heard that Sam was brilliant, but hearing someone articulate both the business and technology value of APIs the way Sam did was a lot like listening to Chris Collinsworth explain line-play during an NFL football broadcast. You think you have some idea how things work and then they show you the "behind the QB angle" and you realize that a new door of complexity has been opened up to you. All I can say is wow!! There is an incredible underground economic environment living behind the mobile devices and apps that have become such a critical aspect of everyone's life.

Cloud Computing and Platform-Based Vulnerabilities

Sat, 11 Feb 2012 05:00:00 EST

What do these two vulnerabilities have in common? Apache Killer. Post of Doom. Right, they’re platform-based vulnerabilities. Meaning they are vulnerabilities peculiar to the web or application server platform upon which applications are deployed. Mitigations for such vulnerabilities generally point to changes in configuration of the platform – limit post size, header value sizes, turn off some value in the associated configuration. But they also have something else in common – risk. And not just risk in general, but risk to cloud providers whose primary value is in offering not just a virtual server but an entire, pre-integrated and pre-configured application deployment stack. Think LAMP, as an example, and providers like Microsoft (Azure) and VMware (CloudFoundry), more commonly adopting the moniker of PaaS. It’s an operational dream to have a virtual server pre-configured and ready to go with the exact application deployment stack needed and offers a great deal of value in terms of efficiency and overall operational investment, but it is – or should be – a security professional’s nightmare. It’s not unlike the recent recall of Chevy Volts – a defect in the platform needs to be mitigated. The only way to do it, for car owners, is to effectively shut down their ability to drive while a patch is applied. It’s disruptive, it’s expensive (you still have to get to work, after all), and it’s frustrating for the consumer. For the provider, it’s bad PR and negatively impacts the brand. Neither of which is appealing.

Who's Managing Your PaaS Apps?

Fri, 10 Feb 2012 09:00:00 EST

PaaS v2.0 should be more open than the current implementations, and cultivate tools communities. But the focus on open development stacks is ignoring the second aspect of PaaS - the management of live applications after they are built. PaaS providers need to allow for communication of SLA and business process requirements by consumers, and cloud management tools can help. We are also seeing the realization of the application-centric approach to cloud management. That takes me to the part that no one is really talking about when it comes to PaaS - the management aspect. Up to this point, we have more often talked about the development side (pre-production). But, PaaS then hosts the application created. How will organizations dictate the SLAs to be in place when the application is live for others to consume (i.e. scale-up, scale-down, etc.)? How will they relay their DR requirements, or moreover how will PaaS providers allow for different RTO/RPO strategies?

NSF Releases Cloud Computing Report

Thu, 09 Feb 2012 11:30:00 EST

The National Science Foundation released their report on cloud computing. It can be found here. The intent of this report is to provide information that guides funding programs. The NSF used NIST’s guidance on cloud computing to inform their research and decision making. This report will be instrumental in informing Federal decision makers and cloud investment. It is important to pay attention to such documents as they can help businesses align with Federal government priorities. The NSF has identified the following areas as research areas worthy of financial support from the federal government:

Enterprise 2.0 Cloud Computing – DevOps for Everyone

Thu, 09 Feb 2012 05:30:00 EST

Although it can feel like you’re playing an intense game of Buzzword Bingo, the key way to approach new technologies like Cloud Computing is to marry them up with other hot topics, like social media and big data. Typically these aren’t entirely different domains more so simply different perspectives on to the same system, and so it’s helpful to throw them all in a pot and discuss: DevOps, IaaS, PaaS, web 2.0, enterprise 2.0, .. etc.

EMC VFCache Respinning SSD and Intelligent Caching (Part 2)

Wed, 08 Feb 2012 16:15:00 EST

EMC vFCache respinning SSD and intelligent caching (Part II)

By Greg Schulz

This is the second of a two part series pertaining to EMC VFCache, you can read the first part here.

In this part of the series, lets look at some common questions along with comments and perspectives.

Common questions, answers, comments and perspectives:

Why would EMC not just go into the same market space and mode as FusionIO, a model that many other vendors seam eager to follow? IMHO many vendors are following or chasing FusionIO thus most are selling in the same way perhaps to the same customers. Some of those vendors can very easily if they were not already also make a quick change to their playbook adding some new moves to reach broader audience. Another smart move here is that by taking a companion or complimentary approach is that EMC can continue selling existing storage systems to customers, keep those investments while also supporting competitors products. In addition, for those customers who are slow to adopt the SSD based techniques, this is a relatively easy and low risk way to gain confidence. Granted the disk drive was declared dead several years (and yes also several decades) ago, however it is and will stay alive for many years due to SSD helping to close the IO storage and performance gap.

Data center and storage IO performance capacity gap (Courtesy of Cloud and Virtual Data Storage Networking (CRC Press))

Has this been done before? There have been other vendors who have done LUN caching appliances in the past going back over a decade. Likewise there are PCIe RAID cards that support flash SSD as well as DRAM based caching. Even NetApp has had similar products and functionality with their PAM cards.

Does VFCache work with other PCIe SSD cards such as FusionIO? No, VFCache is a combination of software IO intercept and intelligent cache driver along with a PCIe SSD flash card (which could be supplied as EMC has indicated from different manufactures). Thus VFCache to be VFCache requires the EMC IO intercept and intelligent cache software driver.

Does VFCache work with other vendors storage? Yes, Refer to the EMC support matrix, however the product has been architected and designed to install and coexist into a customers existing environment which means supporting different EMC block storage systems as well as those from other vendors. Keep in mind that a main theme of VFCache is to compliment, coexist, enhance and protect customers investments in storage systems to improve their effectiveness and productivity as opposed to replacing them.

Does VFCache introduce a new point of vendor lockin or stickiness? Some will see or place this as a new form of vendor lockin, others assuming that EMC supports different vendors storage systems downstream as well as offer options for different PCIe flash cards and keeps the solution affordable will assert it is no more lockin that other solutions. In fact by supporting third party storage systems as opposed to replacing them, smart sales people and marketeers will place VFCache as being more open and interoperable than some other PCIe flash card vendors approach. Keep in mind that avoiding vendor lockin is a shared responsibility (read more here).

Does VFCache work with NAS? VFCache does not work with NAS (NFS or CIFS) attached storage.

Does VFCache work with databases? Yes, VFCache is well suited for little data (e.g. database) and traditional OLTP or general business application process that may not be covered or supported by other so called big data focused or optimized solutions. Refer to this EMC document (and this document here) for more information.

Does VFCache only work with little data? While VFCache is well suited for little data (e.g. databases, share point, file and web servers, traditional business systems) it also able to work with other forms of unstructured data.

Does VFCache need VMware? No, While VFCache works with VMware vSphere including a vCenter plug in, however it does not need a hypervisor and is practical in a physical machine (PM) as it is in a virtual machine (VM).

Does VFCache work with Microsoft Windows? Yes, Refer to the EMC support matrix for specific server operating systems and hypervisor version support.

Does VFCache work with other unix platforms? Refer to the EMC support matrix for specific server operating systems and hypervisor version support.

How are reads handled with VFCache? The VFCache software (driver if you prefer) intercepts IO requests to LUNs that are being cached performing a quick lookup to see if there is a valid cache entry in the physical VFCache PCIe card. If there is a cache hit the IO is resolved from the closer or local PCIe card cache making for a lower latency or faster response time IO. In the case of a cache miss, the VFCache driver simply passes the IO request onto the normal SCSI or block (e.g. iSCSI, SAS, FC, FCoE) stack for processing by the downstream storage system (or appliance). Note that when the requested data is retrieved from the storage system, the VFCache driver will based on caching algorithms determinations place a copy of the data in the PCIe read cache. Thus the real power of the VFCache is the software implementing the cache lookup and cache management functions to leverage the PCIe card that complements the underlying block storage systems.

How are writes handled with VFCache? Unless put into a write cache mode which is not the default, VFCache software simply passes the IO operation onto the IO stack for downstream processing by the storage system or appliance attached via a block interface (e.g. iSCSI, SAS, FC, FCoE). Note that as part of the caching algorithms, the VFCache software will make determinations of what to keep in cache based on IO activity requests similar to how cache management results in better cache effectiveness in a storage system. Given EMCs long history of working with intelligent cache algorithms, one would expect some of that DNA exists or will be leveraged further in future versions of the software. Ironically this is where other vendors with long cache effectiveness histories such as IBM, HDS and NetApp among others should also be scratching their collective heads saying wow, we can or should be doing that as well (or better).

Can VFCache be used as a write cache? Yes, while its default mode is to be used as a persistent read cache to compliment server and application buffers in DRAM along with enhance effectiveness of downstream storage system (or appliances) caches, VFCache can also be configured as a persistent write cache.

Does VFCache include FAST automated tiering between different storage systems? The first version is only a caching tool, however think about it a bit, where the software sits, what storage systems it can work with, ability to learn and understand IO paths and patterns and you can get an idea of where EMC could evolve it to, similar to what they have done with recoverpoint among other tools.

Evolving data access patterns and life cycles (more retention and reads)

Does VFCache mean all or nothing approach with EMC? While the complete VFCache solution comes from EMC (e.g. PCIe card and software), the solution will work with other block attached storage as well as existing EMC storage systems for investment protection.

Does VFCache support NAS based storage systems? The first release of VFCache only supports block based access, however the server that VFCache is installed in could certainly be functioning as a general purpose NAS (NFS or CIFS) server (see supported operating systems in EMC interoperability notes) in addition to being a database or other other application server.

Does VFCache require that all LUNs be cached? No, you can select which LUNs are cached and which ones are not.

Does VFCache run in an active / active mode? In the first release it is active passive, refer to EMC release notes for details.

Can VFCache be installed in multiple physical servers accessing the same shared storage system? Yes, however refer to EMC release notes on details about active / active vs. active / passive configuration rules for ensuring data integrity.

Who else is doing things like this? There are caching appliance vendors as well as others such as NetApp and IBM who have used SSD flash caching cards in their storage systems or virtualization appliances. However keep in mind that VFCache is placing the caching function closer to the application that is accessing it there by improving on the locality of reference (e.g. storage and IO effectiveness).

Does VFCache work with SSD drives installed in EMC or other storage systems? Check the EMC product support matrix for specific tested and certified solutions, however in general if the SSD drive is installed in a storage system that is supported as a block LUN (e.g. iSCSI, SAS, FC, FCoE) in theory it should be possible to work with VFCache. Emphasis, visit the EMC support matrix.
What type of flash is being used?

What type of nand flash SSD memory is EMC using in the PCIe card? The first release of VFCache is leveraging enterprise class SLC (Single Level Cell) nand flash which has been used in other EMC products for its endurance, long duty cycle to minnimize or eliminate concerns of wear and tear while meeting read and write performance. EMC has indicated that they will also as part of an industry trend leverage MLC along with Enterprise MLC (EMLC) technologies on a go forward basis.

Doesnt nand ssd flash cache wear out? While nand flash SSD can wear out over time due to extensive write use, the VFCache approach mitigates this by being primarily a read cache reducing the number or program / erase cycles (P/E cycles) that occur with write operations as well as initially leveraging longer duty cycle SLC flash. EMC also has several years experience from implementing wear leveling algorithms into the storage systems controllers to increase duty cycle and reduce wear on SLC flash which will play forward as MLC or Enterprise MLC (EMLC) techniques are leveraged. This differs from vendors who are positioning their SLC or MLC based flash PCIe SSD cards for mainly write operations which will cause more P/E cycles to occur at a faster rate reducing the duty or useful life of the device.

How much capacity does the VFCache PCIe card contain? The first release supports a 300GB card and EMC has indicated that added capacity and configuration options are in their plans.

Does this mean disks are dead? Contrary to popular industry folk lore (or wish) the hard disk drive (HDD) has plenty of life left part of which has been increased by being complimented by VFCache.

Various SSD locations, types, packaging and usage scenario options

Can VFCache work in blade servers? The VFCache software is transparent to blade, rack mount, tower or other types of servers. The hardware part of VFCache is a PCIe card which means that the blade server or system will need to be able to accommodate a PCIe card to compliment the PCIe based mezzaine IO card (e.g. iSCSI, SAS, FC, FCOE) used for accessing storage. What this means is that for blade systems or server vendors such as IBM who have a PCIe expansion module for their H series blade systems (it consumes a slot normally used by a server blade), PCIe cache cards like those being initially released by IBM could work, however check with the EMC interoperability matrix, as well as your specific blade server vendor for PCIe expansion capabilities. Given that EMC leverages Cisco UCS for their vBlocks, one would assume that those systems will also see VFCache modules in those systems. NetApp partners with Cisco using UCS in their FlexPods so you see where that could go as well along with potential other server vendors support including Dell, HP, IBM and Oracle among others.

What about benchmarks? EMC has released some technical documents that show performance improvements in Oracle environments such as this here. Hopefully we will see EMC also release other workloads for different applications including Microsoft Exchange Solutions Proven (ESRP) along with SPC similar to what IBM recently did with their systems among others.

How do the first EMC supplied workload simulations compare vs. other PCIe cards? This is tough to gauge as many SSD solutions and in particular PCIe cards are doing apples to oranges comparisons. For example to generate a high IOPs rating for marketing purposes, most SSD solutions are stress performance tested at 512 bytes or 1/2 of a KByte or at least 1/8 of a small 4Kbyte IO. Note that operating systems such as Windows are moving to 4Kbyte page allocation size to align with growing IO sizes with databases moving from the old average of 4Kbytes to 8Kbytes and larger. What is important to consider is what is the average IO size and activity profile (e.g. reads vs. writes, random vs. sequential) for your applications. If your application is doing ultra small 1/2 Kbyte IOs, or even smaller 64 byte IOs (which should be handled by better application or file system caching in DRAM), then the smaller IO size and record setting examples will apply. However if your applications are more mainstream or larger, then those smaller IO size tests should be taken with a grain of salt. Also keep latency in mind that many target or oppourtunity applications for VFCache are response time sensitive or can benefit by the improved productivity they enable.

What is locality of reference? Locality of reference refers to how close data is to where it is being requested or accessed from. The closer the data to the application requesting the faster the response time or quick the work gets done. For example in the figure below L1/L2/L3 on board processor caches are the fastest, yet smallest while closest to the application running on the server. At the other extreme further down the stack, storage becomes large capacity, lower cost, however lower performing.

What does cache effectiveness vs. cache utilization mean? Cache utilization is an indicator of how much the available cache capacity is being used however it does not give an indicator of if the cache is being well used or not. For example, cache could be 100 percent used, however there could be a low hit rate. Thus cache effectiveness is a gauge of how well the available cache is being used to improve performance in terms of more work being done (IOPS or bandwidth) or lower of latency and response time.

Isnt more cache better? More cache is not better, it is how the cache is being used, this is a message that I would be disappointed in HDS if they were not to bring up as a point of messaging (or rebuttal) given their history of emphasis cache effectiveness vs. size or quantity (Hu, that is a hint btw ;).

What is the performance impact of VFCache on the host server? EMC is saying greatest of 5 percent or less CPU consumption which they claim is several times less than the competitions worst scenario, as well as claiming 512MB to 1GB of DRM on the server vs. several times that of their competitors. The difference could be expected to be via more off load functioning including flash translation layer (FTL), wear leveling and other optimization being handled by the PCIe card vs. being handled in the servers memory and using host server CPU cycles.

How does this compare to what NetApp or IBM does? NetApp, IBM and others have done caching with SSD in their storage systems, or leveraging third party PCIe SSD cards from different vendors to be installed in servers to be used as a storage target. Some vendors such as LSI have done caching on the PCIe cards (e.g. CacheCaid which in theory has a similar software caching concept to VFCache) to improve performance and effectiveness across JBOD and SAS devices.

What about stale (old or invalid) reads, how does VFCache handle or protect against those? Stale reads are handled via the VFCache management software tool or driver which leverages caching algorithms to decide what is valid or invalid data.

How much does VFCache cost? Refer to EMC announcement pricing, however EMC has indicated that they will be competitive with the market (supply and demand).

If a server shutdowns or reboots, what happens to the data in the VFCache? Being that the data is in non volatile SLC nand flash memory, information is not lost when the server reboots or loses power in the case of a shutdown, thus it is persistent. While exact details are not know as of this time, it is expected that the VFCache driver and software do some form of cache coherency and validity check to guard against stale reads or discard any other invalid cache entries.

What will EMC do with VFCache in the future and on a larger scale such as an appliance? EMC via its own internal development and via acquisitions has demonstrated ability to use various clustered techniques such as RapidIO for VMAX nodes, InfiniBand for connecting Isilon nodes. Given an industry trend with several startups using PCIe flash cards installed in a server that then functions as a IO storage system, it seems likely given EMCs history and experience with different storage systems, caching, and interconnects that they could do something interesting. Perhaps Oracle Exadata III (Exadata I was HP, Exadata II was Sun/Oracle) could be an EMC based appliance (That is pure speculation btw)?

EMC has already shown how it can use SSD drives as a cache extension in VNX and CLARiiON servers ( FAST CACHE ) in addition to as a target or storage tier combined with Fast for tiering. Given their history with caching algorithms, it would not be surprising to see other instantiations of the technology deployed in complimentary ways.

Finally, EMC is showing that it can use nand flash SSD in different ways, various packaging forms to apply to diverse applications or customer environments. The companion or complimentary approach EMC is currently taking contrasts with some other vendors who are taking an all or nothing, its all SSD as disk is dead approach. Given the large installed base of disk based systems EMC as well as other vendors have in place, not to mention the investment by those customers, it makes sense to allow those customers the option of when, where and how they can leverage SSD technologies to coexist and complement their environments. Thus with VFCache, EMC is using SSD as a cache enabler to discuss the decades old and growing storage IO to capacity performance gap in a force multiplier model that spreads the cost over more TBytes, PBytes or EBytes while increasing the overall benefit, in other words effectiveness and productivity.

Ok, nuff said for now, I think I see some storm clouds rolling in...

Cheers Gs

Greg Schulz - Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

Amazon S3’s Exponential Growth

Wed, 08 Feb 2012 13:47:00 EST

In a recent Amazon Web Service Blog, it was quoted that Amazon S3 has reached over 762 Billion objects at the end of 2011. We have been following Amazon S3’s growth closely. As usual, we will plug the numbers in an Excel spread-sheet and see its growth in a chart. As shown in the chart, you can see the numbers fit into an exponential trend line pretty well. Worked out the math, the Amazon S3 objects double every 8 months. If you never used Amazon S3 before, It is OK because you may never need to directly interface Amazon S3. You may be using Amazon S3 indirectly through some online storage services, where quite many are powered by Amazon S3.

API Management – Infrastructure vs SaaS

Wed, 08 Feb 2012 13:15:00 EST

The Enterprise is buzzing with API initiatives these days. APIs not only serve mobile applications, they are increasingly redefining how the enterprise does B2B and integration in general. API management as a category follows different models. On one hand, certain technology vendors offer specialized infrastructure to handle the many aspects of API management. On the other, an increasing number of SaaS vendors offer a service which you subscribe to, providing a pre-installed, hosted, basic API management system. Hybrid models are emerging, but that’s a topic for a future post. Before opting for a pure SaaS-based API management solution offering, consider these below.

An Analysis of a NASA Dbase Hack-and-Dump

Wed, 08 Feb 2012 09:30:00 EST

[Editor's note: this analysis predates any official announcements by NASA] Recently, some news of a NASA hack-and-dump passed my twitter deck. I decided after watching a few of my friends re-tweet the news that it might be worth checking out. At least I’d see if I could perform some password analysis on any dumped credentials, or even test out the new Crucialpoint Cloudera Hadoop password cracker on any leftover hashes. What follows is a step-by-step methodology of how I put together the pieces on the hack, and what I think likely happened based on my experience as a web application hacker/programmer.

Three Buzzwords That Every CIO Hears but One They Should Listen To

Wed, 08 Feb 2012 05:00:00 EST

Anyone that's managing an enterprise IT with aging or outdated client/server systems is starting to feel the heat. Soon, their systems and applications will be obsolete and unsupported. At the same time, the industry as whole is seeking the fastest gateway to the latest .NET, HTML5 and mobile deployments including SaaS models. Right about now, there will be a knock on your door with team members offering their advice as to what you should do to ‘keep up' with the latest trends and supported architectures. Here are three of the buzzwords they will have probably used, and what they actually mean.

OpenStack Cloud Pulls Plug on Microsoft Hyper-V

Wed, 08 Feb 2012 04:08:00 EST

Despite Microsoft's best efforts to "engineer compatibility" between its Hyper-V virtualisation software and the OpenStack open source cloud project since October 2010, it appears that may have run out for the Redmond software giant. Adrian Bridgwater

Getting the Best Out of Amazon EC2 Micro Instances

Tue, 07 Feb 2012 22:25:00 EST

One of the popular instance types supported by Amazon EC2 is the Micro Instance. In November 2010, Amazon announced the free tier and started offering 750 hours of Micro Instance usage free per month for the first one year. Initially this offer was available only to Amazon Linux instances but last month AWS announced that they are extending this to Microsoft Windows Server instances. Looking at the technical specifications, the Micro Instance type doesn’t have the muscle required for heavy lifting. It comes with burst CPU capacity that can go up to 2 Elastic Compute Units (ECU). That means the CPU performance is not predictable. The main memory offered is 613MB! This is just not sufficient for running any serious workloads. Of course, storage can be added through EBS and the free tier covers up to 30GB of storage.

EMC VFCache Respinning SSD and Intelligent Caching (Part I)

Tue, 07 Feb 2012 17:00:00 EST

This is the first part of a two part series covering EMC VFCache. EMC formerly announced VFCache (aka Project Lightning) an IO accelerator product that comprises a PCIe nand flash card (aka Solid State Device or SSD) and intelligent cache management software. In addition EMC is also talking about the next phase of the flash business unit and project Thunder. The approach EMC is taking with vFCache should not be a surprise given their history of starting out with memory and SSD evolving it into an intelligent cache optimized storage solution.

Software Defined Networks (SDNs) - A History Lesson

Tue, 07 Feb 2012 10:15:00 EST

I'm a big fan of history, especially technology history, because it gives us such wonderful guidance about what to expect with each new "technology disruption". Just like the history of mankind, technology tends to follow repeatable trends, actions and mistakes. The latest trend that I've been carefully watching is around Software Defined Networks (SDNs). Today's SDN discussions are primarily focused on how new paradigms will change the architecture of IP networks and how network-level services are delivered and managed. This shift is being led by companies like Big Switch, Nicira, Embrane

Desktop VDI May Be Ready for Prime Time but Is the Network?

Tue, 07 Feb 2012 08:45:00 EST

Considering the innate differences between just the two most popular mobile operating systems – Android and iOS – gives rise to understanding how costly and complex an infrastructure might need to be to support both. It’s not at all unlike the issues with server virtualization. Management and delivery architectures require different solutions depending on the platform, so despite potentially costly investments to scale, organizations are often staying single-vendor with respect to its virtualization platform strategy.

How Fast Does Technology Change?

Tue, 07 Feb 2012 07:30:00 EST

How fast does technology change? What is its impact? It is something like the hour hand of a clock. If you keep staring at it you would feel that it always remains stationary. If you go away and come back after sometime, you will see that it has moved a lot. The same thing is true with technology. “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.” – Roy Amara was a researcher, scientist and past president of the Institute for the Future.

Cross-Platform Mobile Code Generator – a Tool Comparison

Tue, 07 Feb 2012 05:19:00 EST

This is the fourth post for cross-platform mobile development tool comparison. There are two more to come. For convenience of analysis, I had divided the tools into five categories (here is an overview). Mobile Web (JavaScript-CSS library), (here is the detail review) Visual Tool (No access to Code), (here is the detail review) App Generator (Native application for multiple platforms), Hybrid App (Leverages embedded browser control) and Game Builder. The classification is somewhat arbitrary and for some tools it becomes little difficult to classify but here is my logic.

How Quickly Will Software Vendors Move to the Cloud?

Tue, 07 Feb 2012 05:15:00 EST

There’s an excellent discussion going on over on the Cloud Computing Google Group about the pace of migration of traditional software to a SaaS model. Here I recently went into some of the very real reasons why the migration is slower than some would like, but didn’t really talk about the pace of adoption. There are some numbers that make for some interesting analysis. According to PwC, in 2009, the top 100 software vendors (traditional non-SaaS) generated 3.7% of their revenues from SaaS in the US; and 1.1% of their revenues from SaaS in Europe. In the same report, the US has a 44% market share and Europe has 36% market share by revenue (License, Maintenance and Support). According to Gartner, in 2010, the WW installed enterprise software market grossed about $104 Billion. So, roughly, we could say that installed software vendors (US & EU only) brought in nearly $5 Billion in revenues in 2010. So nearly 5% of revenues since the inception of SaaS (not including ASP)?

Organizing the Private Cloud: Coming Solutions

Tue, 07 Feb 2012 04:30:00 EST

There was a time in the data center when we dedicated specific infrastructure and specific administrators to specific projects. This was a period of tremendous inefficiency and underutilization. This approach created a huge overhead in terms of administration, and project cycles that seemed to go on and on.

Today, however, cloud computing and virtualization technologies have change things for the better. Data centers are much more likely to pool resources – computing, storage, and network resources – into a private cloud configuration.

Private cloud challenges

While these virtualized private cloud solutions don’t have the administrative headaches that the old approach had, they do present some unique challenges. There are a number of components in the private cloud that need to be correctly managed, including:

• Data center automation

• Virtualization management

• Performance monitoring

• Automated provisioning

• Chargebacks

Each of these areas requires a separate product or solution. In some ways, this recreates some of the inefficiencies of the pre-virtualization world. Each component acts independently, and it’s up to data center personnel to try to fit the pieces together and make sure each solution is working well with the others.

Coming solutions

Fortunately, solutions providers are beginning to recognize this problem. Solutions are emerging that look at the private cloud from the bird’s eye view, and that manage all of the various moving parts at once.

A good example of this is OpenStack. OpenStack is an open source project that is designed to provide a core set of management tools to help organize the private cloud. Some of those tools include things like management of virtual machines, storage management, and even image servicing.

OpenStack was originally developed by NASA along with Rackspace, but is now an open source project with more than 130 companies participating. These are big names in cloud computing, too, and include companies like Cisco, Citrix, HP, NEC, and others.

Making the private cloud even better

In the long run, these solutions are going to offer an even greater degree of efficiency and management capability to the private cloud. Once they take hold and start to gain third party support, they’ll become viable solutions for just about any data center.

Advanced Load Balancing for Developers

Mon, 06 Feb 2012 09:45:00 EST

It has been a while since I wrote an installment of Load Balancing for Developers, and now I think it has been too long, but never fear, this is the grad-daddy of Load Balancing for Developers blogs, covering a useful bit of information about Application Delivery Controllers that you might want to take advantage of. For those who have joined us since my last installment, feel free to check out the entire list of blog entries (along with related blog entries) here, though I assure you that this installment, like most of the others, does not require you to have read those that went before.

ZapNGo! Is still a growing enterprise, now with several dozen complex applications and a high availability architecture that spans datacenters and the cloud. While the organization relies upon its web properties to generate revenue, those properties have been going along fine with your Application Delivery Controller (ADC) architecture.

Now though, you’re seeing a need to centralize administration of a whole lot of functions. What worked fine separately for one or two applications is no longer working so well now that you have several development teams and several dozen applications, and you need to find a way to bring the growing inter-relationships under control before maintenance and hidden dependencies swamp you in a cascading mess of disruption.

With maintenance taking a growing portion of your application development manhours, and a reasonably well positioned test environment configured with a virtual ADC to mimic your production environment, all you need now is a way to cut those maintenance manhours and reduce the amount of repetitive work required to create or update an application. Particularly update an application, because that is a constant problem, where creating is less frequent.

With many of the threats that your ZapNGo application will be known as ZapNGone eliminated, now it is efficiencies you are after. And believe it or not, these too are available in an ADC. Not all ADC’s are created equal, but this discussion will stay on topics that most ADCs can handle, and I’ll mention it when I stray from generic into specific – which I will do in one case because only one vendor supports one of the tools you can use, but all of the others should be supported by whatever ADC vendor you have, though as always, check with your vendor directly first, since I’m not an expert in the inner workings of every one.

There is a lot that many organizations do for themselves, and the array of possibilities is long – from implementing load balancing in source code to security checks in the application, the boundaries of what is expected of developers are shaped by an organization, its history, and its chosen future direction. At ZapNGo, the team has implemented a virtual test environment that as close as possible mirrors production, so that code can be implemented and tested in the way it will be used. They use an ADC for load balancing, so that they don’t have to rewrite the same code over and over, and they have a policy of utilizing a familiar subset of ADC functionality on all applications that face the public.

The company is successful and growing, but as always happens in companies in that situation, the pressures upon them are changing just by virtue of their growth. There are more new people who don’t yet have intimate knowledge of the code base, network topology, security policies, whatever their area of expertise is. There are more lines of code to maintain, while new projects are being brought up at a more rapid pace and with higher priorities (I’ve twice lived through the “Everything is high priority? Well this is highest priority!” syndrome while working in IT. Thankfully, most companies grow out of that fast when it’s pointed out that if everything is priority #1, nothing is). Timelines to complete projects – be they new development, bug fixes, or enhancements are stretching longer and longer as the percentage of gurus in the company is down and the complexity of the code and the architecture it runs on is up.

So what is a development manager to do to increase productivity? Teaming newer developers with people who’ve been around since the beginning is helping, but those seasoned developers are a smaller and smaller percentage of the workforce, while the volume of work has slowly removed them from some of the many products now under management. Adopting coding standards and standardized libraries helps increase experience portability between projects, but doesn’t do enough.

Enter offloading to the ADC. Some things just don’t have to be done in code, and if they don’t have to be, at this stage in the company’s growth, IT management at ZapNGo (that’s you!) decides they won’t be. There just isn’t time for non-essential development anymore.

Utilizing a policy management tool and/or an Application Firewall on the ADC can improve security without increasing the code base, for example. And that shaves hours off of maintenance projects, while standardizing on one or a few implementations that are simply selected on the ADC. Implementing Web Application Acceleration protocols on the ADC means that less in-code optimization has to occur. Performance is no longer purely the role of developers (but of course it is still a concern. No Web Application Acceleration tool can make a loop that runs for five minutes run faster), they can allow the Web Application Acceleration tool to shrink the amount of data being sent to the users’ browser for you. Utilizing a WAN Optimization ADC tool to improve the performance of bulk copies or backups to a remote datacenter or cloud storage… The list goes on and on.

The key is that the ADC enables a lot of opportunities for App Dev to be more responsive to the needs of the organization by moving repetitive tasks to the ADC and standardizing them. And a heaping bonus is that it also does that for operations with a different subset of functionality, meaning one toolset gives both App Dev and Operations a bit more time out of their day for servicing important organizational needs. Some would say this is all part of DevOps, some would say it is not. I leave those discussions to others, all I care is that it can make your apps more secure, fast, and available, while cutting down on workload.

And if your ADC supports an SSL VPN, your developers can work from home when necessary. Or more likely, if your code is your IP, a subset of your developers can. Making ZapNGo more responsive, easier to maintain, and more adaptable to the changes coming next week/month/year. That’s what ADCs do. And they’re pretty darned good at it.

That brings us to the one bit that I have to caveat with F5 only, and that is iApps. An iApp is a constructed configuration tool that asks a few questions and then deploys all the bits necessary to set up an ADC for a particular application. Why do I mention it here? Well if you have dozens of applications with similar characteristics, you can create an iApp Template and use it to rapidly bring new applications or new instances of applications online. And since it is abstracted, these iApp templates can be designed such that AppDev, or even the business owner, is able to operate them Meaning less time worrying about what network resources will be available, how they’re configured, and waiting for operations to have time to implement them (in an advanced ADC that is being utilized to its maximum in a complex application environment, this can be hundreds of networking objects to configure – all encapsulated into a form). Less time on the project timeline, more time for the next project. Or for the post deployment party. One of the two. That’s it for the F5 only bit.

And knowing that all of these items are standardized means less things to get mis-configured, more surety that it will all work right the first time. As with all of these articles, that offers you the most important benefit… A good night’s sleep.

Technorati Tags: Application Delivery Controllers,VPN,Security,Applicaiton Development,Acceleration,WAN Optimization,Encryption,F5 Networks,Load Balancing For Developers,Don MacVittie,blog

Connect with Don:	Connect with F5:

Has Cloud Finally "Crossed the Chasm" in IT?

Mon, 06 Feb 2012 06:15:00 EST

Every year, our friends at ESG post results of their annual Spending Intentions Survey, indicating where many businesses are likely to spend their IT dollars over the coming year. Recently Steve Duplessie posted an article on his blog entitled Cloud – The Cost Containment Strategy that concludes cloud has finally “crossed the chasm” in IT. According to preliminary data, cloud represents the largest % projected spending increase for 2012 IT initiatives– a very exciting turn. Truth is, cloud storage addresses long-standing IT priorities, with three of these priorities topping the list nearly every year.

Installed Software Leverages Cloud Computing, Not Replaced by It

Mon, 06 Feb 2012 05:30:00 EST

Explore why not just revenue streams and business models are slowing the demise of installed software. And see how independent software vendors and their customers are leveraging the cloud....

Cloud Management: The Agentless Approach

Mon, 06 Feb 2012 03:00:00 EST

Automating the deployment and runtime management of large application deployments running on multiple cloud resources across various cloud providers is a challenging problem. Managing remote resources is not a new problem the big difference is that in cloud the resources are added and removed dynamically and at a larger scale. Before cloud people have built management systems using two approaches:

Agent
Agentless

Historically the major trade-off between the agent and the agentless approach has been control vs. rollout time and maintenance costs. You usually get more control with the agent approach compared to the agentless approach. Whereas, the agentless approach is easy to deploy and manage as it doesn’t require rolling out new agents and maintaining their versions.

At Kaavo for deploying software and performing runtime configuration we choose to use SSH instead of requiring proprietary agents to be installed on the managed cloud resources. Some of the design considerations for our decision are:

Easier Rollout and Ongoing Maintenance: Using SSH for management gives you same or more control and security compared to an agent based approach without the overhead of deploying and maintaining proprietary agents on all images across all supported datacenters.
Better Security: Both the agent approach vs. the agentless (SSH) requires communication between the group of manager servers and the servers/resources being managed. As a result you have to manage the firewall rules on the communication ports (for incoming and outgoing packets) properly on the cloud servers/resources to avoid holes for the intruders to exploit. However, since SSH protocol has been around for a while and has been well tested on a large scale it is less likely to have any unknown security vulnerability compared to writing your own proprietary agent or protocol.
Greater Control and Flexibility: In agent based approach the agent code that is executed on the servers or the server side scripts can’t be changed on the fly in case there is any unexpected change in the environment. Whereas in our agentless approach we generate the configuration scripts and files just in time and send them to the servers for execution. This on-demand just-in-time generation of scripts gives us greater flexibility and control in managing cloud resources.

To rollout a scalable agentless approach for cloud management we had to solve some interesting complex problems like managing firewall rules automatically for managed servers in a dynamic cloud environment, handling distributed event queues, and handling execution order dependencies for parallel processes. We solved these hard problems because we didn’t want to take the easy way out by using configuration agents. If you have any further questions about this or want to learn more about why we choose agentless approach for cloud management please contact us.