Welcome!

Linux Containers Authors: Liz McMillan, Yeshim Deniz, Pat Romanski, Derek Weeks, Elizabeth White

Related Topics: Open Source Cloud, Linux Containers

Open Source Cloud: Article

Strengthening Open Source's Weakest Link: Software Testing

Broader, more collaborative open testing can yield more meaningful results for business and developers alike

Murugan Pal (pictured), founder and CTO of SpikeSource, writes: Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how much of the application has been tested?

The answer is probably closer to 30 percent than 50 percent, since both users probably focused on common functions like start-up, shutdown, and data access. The problem gets amplified if the application is built for n-tier deployment based on service-oriented architecture. newspapers and broadcasters. The service handles between 150,000 and 500,000 pages of content per affiliate per day, supporting 11,000 concurrent users. MySQL, a free open source database, has been the backbone of AP Hosted News since 2002.

Everyone knows that the cornerstone of open source software is the free availability of its source code, which lets developers and users around the world contribute to it and improve it. The software naturally becomes stronger as it accumulates improvements and sheds imperfections. The quality improves based on more usage and reviews.

But the model breaks down when it comes to making sure the software actually works in real-world deployment scenarios. The power of participation has been confined almost entirely to the development phase of the software life cycle. Testing remains open source's weakest link as it is difficult to reproduce all intended usages.

While code repositories and other shared resources help developers revise and build upon the efforts of their peers, the testing of that software has remained an uncoordinated, isolated affair. Instead of learning from and enhancing each other's tests, users and developers test the same functions and routines, and have no way to easily share their results with each other.

Because most testers are testing only on the platform they happen to be using, most test results aren't widely applicable. Results that show how well a piece of software works on a particular platform might not say much about how well it works with a different operating system, or how well it interacts with other software components. That's a major shortcoming, since open source software components are mostly used as part of a stack with other components.

A Moving Target
The constantly changing nature of many open source programs makes meaningful results even harder to come by. To get results that would be accurate and meaningful to a broad section of the open source community, a user would have to constantly retest it on an ever-growing number of platforms (which are also changing).

As a result, some of the biggest challenges of open source software have remained intact. "Dependency hell," (Jar Wars and DLL Hell) in which each piece of software relies on a specific version of another piece of software, continues to be a constant time drain for many IT departments. Not only are the dependencies difficult to resolve, some times you end up with redundant footprints of the same libraries embedded in the integrated runtime (e.g., Log4J in Tomcat, Struts, etc.).

What if the open source development model - the "architecture of participation," to use Tim O'Reilly's phrase - could be extended to software testing? If users could easily access, build upon, and contribute to a growing body of open source tests, testing could become an extension of the participatory development process. Fixes could be validated faster, and functionality and backward compatibility across different versions of integrated software components would be easier. Even enterprise customers can participate in this model by validating their tests on integrated hardware and software runtime environments.

Participatory testing would also help certify the interoperability of the exponentially increasing combinations of component choices. Most businesses use open source software not in isolation, but in stacks of interoperating components. Tests should be able to tell you exactly how well those components work together.

Just as the participatory development community relies on open resources and information repositories for source code, the participatory testing community should have open resources for testing -including open tests, test manifests, test results, and interfaces/protocols to share test results and tests.

That's where SpikeSource want to be a catalyst in promoting participatory testing. We provide all of those resources, as well as an environment for open source software users and developers to share, obtain, and exchange federated information about open source testing.

Businesses can upload an application to SpikeSource's open testing tool, which continuously pulls code from open source repositories and builds its own repository of different versions of different components and operating systems. SpikeSource automatically constructs systems out of those repositories, provisions them on virtualized runtime environments, tests them, and records the results.

Interoperability Is Key
For most businesses, how well a component works with others is just as important as its independent functionality. For example, if there's a change to the Apache servlet engine Tomcat, a test should show how those changes impact other components like JK2, ConnectorJ or MySQL.

The ultimate goal is to make open source software more scalable and predictable, so that business can use it in conjunction with or as an alternative to proprietary software. Our goal at SpikeSource is to help further the enterprise adoption of open source software. Through testing, it becomes more reliable, easy, and safe for enterprises to deploy.

More Stories By Murugan Pal

Murugan Pal is founder and CTO of SpikeSource.

Comments (3) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Jonathan Bruce's Web Log 09/19/05 01:06:38 PM EDT

Trackback Added: Open source components fall short of the quality metrics?; Sys-Con ran an article on September 16th, by Murugan Pal, CTO of SpikeSource.com Strengthening Open Source's Weakest Link: Software Testing — Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how...

David Tomlinson 09/12/05 06:38:50 PM EDT

First, the writer tells us that the biggest problem is testing. Then that the problem with testing is that everyone tests the same thing. They he tells us that everybody tests on their own platform, so my test results will mean nothing to you. Then he talks about jar wars and dependency hell. Well, which is it? Is the biggest problem with open source that the writer's company doesn't have enough of our business? Have I ever seen a more obvious commercial for a vendor? Of course. Do I want to see more? Hell, no!

The only reason for writing this article is to get his company's name out there and get a few hits on their website. I gotta admit they got a hit from me, but it's the last one.

Pointless article.

Enterprise Open Source Magazine 09/12/05 09:52:44 AM EDT

Strengthening Open Source's Weakest Link. Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how much of the application has been tested? The answer is probably closer to 30 percent than 50 percent, since both users probably focused on common functions like start-up, shutdown, and data access.

@ThingsExpo Stories
Artificial intelligence, machine learning, neural networks. We’re in the midst of a wave of excitement around AI such as hasn’t been seen for a few decades. But those previous periods of inflated expectations led to troughs of disappointment. Will this time be different? Most likely. Applications of AI such as predictive analytics are already decreasing costs and improving reliability of industrial machinery. Furthermore, the funding and research going into AI now comes from a wide range of com...
SYS-CON Events announced today that Silicon India has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Published in Silicon Valley, Silicon India magazine is the premiere platform for CIOs to discuss their innovative enterprise solutions and allows IT vendors to learn about new solutions that can help grow their business.
SYS-CON Events announced today that Ayehu will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara California. Ayehu provides IT Process Automation & Orchestration solutions for IT and Security professionals to identify and resolve critical incidents and enable rapid containment, eradication, and recovery from cyber security breaches. Ayehu provides customers greater control over IT infras...
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business...
SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market acr...
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organi...
In this presentation, Striim CTO and founder Steve Wilkes will discuss practical strategies for counteracting fraud and cyberattacks by leveraging real-time streaming analytics. In his session at @ThingsExpo, Steve Wilkes, Founder and Chief Technology Officer at Striim, will provide a detailed look into leveraging streaming data management to correlate events in real time, and identify potential breaches across IoT and non-IoT systems throughout the enterprise. Strategies for processing massive ...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
"MobiDev is a Ukraine-based software development company. We do mobile development, and we're specialists in that. But we do full stack software development for entrepreneurs, for emerging companies, and for enterprise ventures," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
SYS-CON Events announced today that Cloud Academy named "Bronze Sponsor" of 21st International Cloud Expo which will take place October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara, CA. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud com...
SYS-CON Events announced today that TechTarget has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TechTarget storage websites are the best online information resource for news, tips and expert advice for the storage, backup and disaster recovery markets.
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
SYS-CON Events announced today that TMC has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo and Big Data at Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Global buyers rely on TMC’s content-driven marketplaces to make purchase decisions and navigate markets. Learn how we can help you reach your marketing goals.
Amazon started as an online bookseller 20 years ago. Since then, it has evolved into a technology juggernaut that has disrupted multiple markets and industries and touches many aspects of our lives. It is a relentless technology and business model innovator driving disruption throughout numerous ecosystems. Amazon’s AWS revenues alone are approaching $16B a year making it one of the largest IT companies in the world. With dominant offerings in Cloud, IoT, eCommerce, Big Data, AI, Digital Assista...
We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists looked at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deliver...
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 21st Int\ernational Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their ...