Linux Containers Authors: Pat Romanski, Liz McMillan, Yeshim Deniz, Kareen Kircher, Carmen Gonzalez

Related Topics: Open Source Cloud, Linux Containers

Open Source Cloud: Article

Strengthening Open Source's Weakest Link: Software Testing

Broader, more collaborative open testing can yield more meaningful results for business and developers alike

Murugan Pal (pictured), founder and CTO of SpikeSource, writes: Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how much of the application has been tested?

The answer is probably closer to 30 percent than 50 percent, since both users probably focused on common functions like start-up, shutdown, and data access. The problem gets amplified if the application is built for n-tier deployment based on service-oriented architecture. newspapers and broadcasters. The service handles between 150,000 and 500,000 pages of content per affiliate per day, supporting 11,000 concurrent users. MySQL, a free open source database, has been the backbone of AP Hosted News since 2002.

Everyone knows that the cornerstone of open source software is the free availability of its source code, which lets developers and users around the world contribute to it and improve it. The software naturally becomes stronger as it accumulates improvements and sheds imperfections. The quality improves based on more usage and reviews.

But the model breaks down when it comes to making sure the software actually works in real-world deployment scenarios. The power of participation has been confined almost entirely to the development phase of the software life cycle. Testing remains open source's weakest link as it is difficult to reproduce all intended usages.

While code repositories and other shared resources help developers revise and build upon the efforts of their peers, the testing of that software has remained an uncoordinated, isolated affair. Instead of learning from and enhancing each other's tests, users and developers test the same functions and routines, and have no way to easily share their results with each other.

Because most testers are testing only on the platform they happen to be using, most test results aren't widely applicable. Results that show how well a piece of software works on a particular platform might not say much about how well it works with a different operating system, or how well it interacts with other software components. That's a major shortcoming, since open source software components are mostly used as part of a stack with other components.

A Moving Target
The constantly changing nature of many open source programs makes meaningful results even harder to come by. To get results that would be accurate and meaningful to a broad section of the open source community, a user would have to constantly retest it on an ever-growing number of platforms (which are also changing).

As a result, some of the biggest challenges of open source software have remained intact. "Dependency hell," (Jar Wars and DLL Hell) in which each piece of software relies on a specific version of another piece of software, continues to be a constant time drain for many IT departments. Not only are the dependencies difficult to resolve, some times you end up with redundant footprints of the same libraries embedded in the integrated runtime (e.g., Log4J in Tomcat, Struts, etc.).

What if the open source development model - the "architecture of participation," to use Tim O'Reilly's phrase - could be extended to software testing? If users could easily access, build upon, and contribute to a growing body of open source tests, testing could become an extension of the participatory development process. Fixes could be validated faster, and functionality and backward compatibility across different versions of integrated software components would be easier. Even enterprise customers can participate in this model by validating their tests on integrated hardware and software runtime environments.

Participatory testing would also help certify the interoperability of the exponentially increasing combinations of component choices. Most businesses use open source software not in isolation, but in stacks of interoperating components. Tests should be able to tell you exactly how well those components work together.

Just as the participatory development community relies on open resources and information repositories for source code, the participatory testing community should have open resources for testing -including open tests, test manifests, test results, and interfaces/protocols to share test results and tests.

That's where SpikeSource want to be a catalyst in promoting participatory testing. We provide all of those resources, as well as an environment for open source software users and developers to share, obtain, and exchange federated information about open source testing.

Businesses can upload an application to SpikeSource's open testing tool, which continuously pulls code from open source repositories and builds its own repository of different versions of different components and operating systems. SpikeSource automatically constructs systems out of those repositories, provisions them on virtualized runtime environments, tests them, and records the results.

Interoperability Is Key
For most businesses, how well a component works with others is just as important as its independent functionality. For example, if there's a change to the Apache servlet engine Tomcat, a test should show how those changes impact other components like JK2, ConnectorJ or MySQL.

The ultimate goal is to make open source software more scalable and predictable, so that business can use it in conjunction with or as an alternative to proprietary software. Our goal at SpikeSource is to help further the enterprise adoption of open source software. Through testing, it becomes more reliable, easy, and safe for enterprises to deploy.

More Stories By Murugan Pal

Murugan Pal is founder and CTO of SpikeSource.

Comments (3) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Most Recent Comments
Jonathan Bruce's Web Log 09/19/05 01:06:38 PM EDT

Trackback Added: Open source components fall short of the quality metrics?; Sys-Con ran an article on September 16th, by Murugan Pal, CTO of SpikeSource.com Strengthening Open Source's Weakest Link: Software Testing — Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how...

David Tomlinson 09/12/05 06:38:50 PM EDT

First, the writer tells us that the biggest problem is testing. Then that the problem with testing is that everyone tests the same thing. They he tells us that everybody tests on their own platform, so my test results will mean nothing to you. Then he talks about jar wars and dependency hell. Well, which is it? Is the biggest problem with open source that the writer's company doesn't have enough of our business? Have I ever seen a more obvious commercial for a vendor? Of course. Do I want to see more? Hell, no!

The only reason for writing this article is to get his company's name out there and get a few hits on their website. I gotta admit they got a hit from me, but it's the last one.

Pointless article.

Enterprise Open Source Magazine 09/12/05 09:52:44 AM EDT

Strengthening Open Source's Weakest Link. Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how much of the application has been tested? The answer is probably closer to 30 percent than 50 percent, since both users probably focused on common functions like start-up, shutdown, and data access.

@ThingsExpo Stories
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
Big Data has been changing the world. IoT fuels the further transformation recently. How are Big Data and IoT related? In his session at @BigDataExpo, Tony Shan, a renowned visionary and thought leader, will explore the interplay of Big Data and IoT. He will anatomize Big Data and IoT separately in terms of what, which, why, where, when, who, how and how much. He will then analyze the relationship between IoT and Big Data, specifically the drilldown of how the 4Vs of Big Data (Volume, Variety,...
From wearable activity trackers to fantasy e-sports, data and technology are transforming the way athletes train for the game and fans engage with their teams. In his session at @ThingsExpo, will present key data findings from leading sports organizations San Francisco 49ers, Orlando Magic NBA team. By utilizing data analytics these sports orgs have recognized new revenue streams, doubled its fan base and streamlined costs at its stadiums. John Paul is the CEO and Founder of VenueNext. Prior ...
SYS-CON Events announced today that Embotics, the cloud automation company, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Embotics is the cloud automation company for IT organizations and service providers that need to improve provisioning or enable self-service capabilities. With a relentless focus on delivering a premier user experience and unmatched customer support, Embotics is the fas...
Ask someone to architect an Internet of Things (IoT) solution and you are guaranteed to see a reference to the cloud. This would lead you to believe that IoT requires the cloud to exist. However, there are many IoT use cases where the cloud is not feasible or desirable. In his session at @ThingsExpo, Dave McCarthy, Director of Products at Bsquare Corporation, will discuss the strategies that exist to extend intelligence directly to IoT devices and sensors, freeing them from the constraints of ...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, will discuss how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team a...
The Open Connectivity Foundation (OCF), sponsor of the IoTivity open source project, and AllSeen Alliance, which provides the AllJoyn® open source IoT framework, today announced that the two organizations’ boards have approved a merger under the OCF name and bylaws. This merger will advance interoperability between connected devices from both groups, enabling the full operating potential of IoT and representing a significant step towards a connected ecosystem.
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, will discuss the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports. The session will include a working demo and a technical d...
What happens when the different parts of a vehicle become smarter than the vehicle itself? As we move toward the era of smart everything, hundreds of entities in a vehicle that communicate with each other, the vehicle and external systems create a need for identity orchestration so that all entities work as a conglomerate. Much like an orchestra without a conductor, without the ability to secure, control, and connect the link between a vehicle’s head unit, devices, and systems and to manage the ...
What are the successful IoT innovations from emerging markets? What are the unique challenges and opportunities from these markets? How did the constraints in connectivity among others lead to groundbreaking insights? In her session at @ThingsExpo, Carmen Feliciano, a Principal at AMDG, will answer all these questions and share how you can apply IoT best practices and frameworks from the emerging markets to your own business.
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smar...
In past @ThingsExpo presentations, Joseph di Paolantonio has explored how various Internet of Things (IoT) and data management and analytics (DMA) solution spaces will come together as sensor analytics ecosystems. This year, in his session at @ThingsExpo, Joseph di Paolantonio from DataArchon, will be adding the numerous Transportation areas, from autonomous vehicles to “Uber for containers.” While IoT data in any one area of Transportation will have a huge impact in that area, combining sensor...
Machine Learning helps make complex systems more efficient. By applying advanced Machine Learning techniques such as Cognitive Fingerprinting, wind project operators can utilize these tools to learn from collected data, detect regular patterns, and optimize their own operations. In his session at 18th Cloud Expo, Stuart Gillen, Director of Business Development at SparkCognition, discussed how research has demonstrated the value of Machine Learning in delivering next generation analytics to impr...
Successful digital transformation requires new organizational competencies and capabilities. Research tells us that the biggest impediment to successful transformation is human; consequently, the biggest enabler is a properly skilled and empowered workforce. In the digital age, new individual and collective competencies are required. In his session at 19th Cloud Expo, Bob Newhouse, CEO and founder of Agilitiv, will draw together recent research and lessons learned from emerging and established ...
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
Amazon has gradually rolled out parts of its IoT offerings, but these are just the tip of the iceberg. In addition to optimizing their backend AWS offerings, Amazon is laying the ground work to be a major force in IoT - especially in the connected home and office. In his session at @ThingsExpo, Chris Kocher, founder and managing director of Grey Heron, explained how Amazon is extending its reach to become a major force in IoT by building on its dominant cloud IoT platform, its Dash Button strat...
@ThingsExpo has been named the Top 5 Most Influential M2M Brand by Onalytica in the ‘Machine to Machine: Top 100 Influencers and Brands.' Onalytica analyzed the online debate on M2M by looking at over 85,000 tweets to provide the most influential individuals and brands that drive the discussion. According to Onalytica the "analysis showed a very engaged community with a lot of interactive tweets. The M2M discussion seems to be more fragmented and driven by some of the major brands present in the...
We are reaching the end of the beginning with WebRTC, and real systems using this technology have begun to appear. One challenge that faces every WebRTC deployment (in some form or another) is identity management. For example, if you have an existing service – possibly built on a variety of different PaaS/SaaS offerings – and you want to add real-time communications you are faced with a challenge relating to user management, authentication, authorization, and validation. Service providers will w...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
SYS-CON Media announced today that @WebRTCSummit Blog, the largest WebRTC resource in the world, has been launched. @WebRTCSummit Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. @WebRTCSummit Blog can be bookmarked ▸ Here @WebRTCSummit conference site can be bookmarked ▸ Here