Welcome!

Linux Containers Authors: Liz McMillan, Derek Weeks, Pat Romanski, Mano Marks, Elizabeth White

Related Topics: Open Source Cloud, Linux Containers

Open Source Cloud: Article

Strengthening Open Source's Weakest Link: Software Testing

Broader, more collaborative open testing can yield more meaningful results for business and developers alike

Murugan Pal (pictured), founder and CTO of SpikeSource, writes: Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how much of the application has been tested?

The answer is probably closer to 30 percent than 50 percent, since both users probably focused on common functions like start-up, shutdown, and data access. The problem gets amplified if the application is built for n-tier deployment based on service-oriented architecture. newspapers and broadcasters. The service handles between 150,000 and 500,000 pages of content per affiliate per day, supporting 11,000 concurrent users. MySQL, a free open source database, has been the backbone of AP Hosted News since 2002.

Everyone knows that the cornerstone of open source software is the free availability of its source code, which lets developers and users around the world contribute to it and improve it. The software naturally becomes stronger as it accumulates improvements and sheds imperfections. The quality improves based on more usage and reviews.

But the model breaks down when it comes to making sure the software actually works in real-world deployment scenarios. The power of participation has been confined almost entirely to the development phase of the software life cycle. Testing remains open source's weakest link as it is difficult to reproduce all intended usages.

While code repositories and other shared resources help developers revise and build upon the efforts of their peers, the testing of that software has remained an uncoordinated, isolated affair. Instead of learning from and enhancing each other's tests, users and developers test the same functions and routines, and have no way to easily share their results with each other.

Because most testers are testing only on the platform they happen to be using, most test results aren't widely applicable. Results that show how well a piece of software works on a particular platform might not say much about how well it works with a different operating system, or how well it interacts with other software components. That's a major shortcoming, since open source software components are mostly used as part of a stack with other components.

A Moving Target
The constantly changing nature of many open source programs makes meaningful results even harder to come by. To get results that would be accurate and meaningful to a broad section of the open source community, a user would have to constantly retest it on an ever-growing number of platforms (which are also changing).

As a result, some of the biggest challenges of open source software have remained intact. "Dependency hell," (Jar Wars and DLL Hell) in which each piece of software relies on a specific version of another piece of software, continues to be a constant time drain for many IT departments. Not only are the dependencies difficult to resolve, some times you end up with redundant footprints of the same libraries embedded in the integrated runtime (e.g., Log4J in Tomcat, Struts, etc.).

What if the open source development model - the "architecture of participation," to use Tim O'Reilly's phrase - could be extended to software testing? If users could easily access, build upon, and contribute to a growing body of open source tests, testing could become an extension of the participatory development process. Fixes could be validated faster, and functionality and backward compatibility across different versions of integrated software components would be easier. Even enterprise customers can participate in this model by validating their tests on integrated hardware and software runtime environments.

Participatory testing would also help certify the interoperability of the exponentially increasing combinations of component choices. Most businesses use open source software not in isolation, but in stacks of interoperating components. Tests should be able to tell you exactly how well those components work together.

Just as the participatory development community relies on open resources and information repositories for source code, the participatory testing community should have open resources for testing -including open tests, test manifests, test results, and interfaces/protocols to share test results and tests.

That's where SpikeSource want to be a catalyst in promoting participatory testing. We provide all of those resources, as well as an environment for open source software users and developers to share, obtain, and exchange federated information about open source testing.

Businesses can upload an application to SpikeSource's open testing tool, which continuously pulls code from open source repositories and builds its own repository of different versions of different components and operating systems. SpikeSource automatically constructs systems out of those repositories, provisions them on virtualized runtime environments, tests them, and records the results.

Interoperability Is Key
For most businesses, how well a component works with others is just as important as its independent functionality. For example, if there's a change to the Apache servlet engine Tomcat, a test should show how those changes impact other components like JK2, ConnectorJ or MySQL.

The ultimate goal is to make open source software more scalable and predictable, so that business can use it in conjunction with or as an alternative to proprietary software. Our goal at SpikeSource is to help further the enterprise adoption of open source software. Through testing, it becomes more reliable, easy, and safe for enterprises to deploy.

More Stories By Murugan Pal

Murugan Pal is founder and CTO of SpikeSource.

Comments (3) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Jonathan Bruce's Web Log 09/19/05 01:06:38 PM EDT

Trackback Added: Open source components fall short of the quality metrics?; Sys-Con ran an article on September 16th, by Murugan Pal, CTO of SpikeSource.com Strengthening Open Source's Weakest Link: Software Testing — Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how...

David Tomlinson 09/12/05 06:38:50 PM EDT

First, the writer tells us that the biggest problem is testing. Then that the problem with testing is that everyone tests the same thing. They he tells us that everybody tests on their own platform, so my test results will mean nothing to you. Then he talks about jar wars and dependency hell. Well, which is it? Is the biggest problem with open source that the writer's company doesn't have enough of our business? Have I ever seen a more obvious commercial for a vendor? Of course. Do I want to see more? Hell, no!

The only reason for writing this article is to get his company's name out there and get a few hits on their website. I gotta admit they got a hit from me, but it's the last one.

Pointless article.

Enterprise Open Source Magazine 09/12/05 09:52:44 AM EDT

Strengthening Open Source's Weakest Link. Pop quiz: If one open source user tests 30 percent of an application, and another tests 20 percent, how much of the application has been tested? The answer is probably closer to 30 percent than 50 percent, since both users probably focused on common functions like start-up, shutdown, and data access.

@ThingsExpo Stories
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between wh...
"Tintri was started in 2008 with the express purpose of building a storage appliance that is ideal for virtualized environments. We support a lot of different hypervisor platforms from VMware to OpenStack to Hyper-V," explained Dan Florea, Director of Product Management at Tintri, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
The security needs of IoT environments require a strong, proven approach to maintain security, trust and privacy in their ecosystem. Assurance and protection of device identity, secure data encryption and authentication are the key security challenges organizations are trying to address when integrating IoT devices. This holds true for IoT applications in a wide range of industries, for example, healthcare, consumer devices, and manufacturing. In his session at @ThingsExpo, Lancen LaChance, vic...
WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was. You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.
Big Data, cloud, analytics, contextual information, wearable tech, sensors, mobility, and WebRTC: together, these advances have created a perfect storm of technologies that are disrupting and transforming classic communications models and ecosystems. In his session at @ThingsExpo, Erik Perotti, Senior Manager of New Ventures on Plantronics’ Innovation team, provided an overview of this technological shift, including associated business and consumer communications impacts, and opportunities it m...
You have great SaaS business app ideas. You want to turn your idea quickly into a functional and engaging proof of concept. You need to be able to modify it to meet customers' needs, and you need to deliver a complete and secure SaaS application. How could you achieve all the above and yet avoid unforeseen IT requirements that add unnecessary cost and complexity? You also want your app to be responsive in any device at any time. In his session at 19th Cloud Expo, Mark Allen, General Manager of...
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus o...
Big Data engines are powering a lot of service businesses right now. Data is collected from users from wearable technologies, web behaviors, purchase behavior as well as several arbitrary data points we’d never think of. The demand for faster and bigger engines to crunch and serve up the data to services is growing exponentially. You see a LOT of correlation between “Cloud” and “Big Data” but on Big Data and “Hybrid,” where hybrid hosting is the sanest approach to the Big Data Infrastructure pro...
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists peeled away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud enviro...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
"LinearHub provides smart video conferencing, which is the Roundee service, and we archive all the video conferences and we also provide the transcript," stated Sunghyuk Kim, CEO of LinearHub, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Things are changing so quickly in IoT that it would take a wizard to predict which ecosystem will gain the most traction. In order for IoT to reach its potential, smart devices must be able to work together. Today, there are a slew of interoperability standards being promoted by big names to make this happen: HomeKit, Brillo and Alljoyn. In his session at @ThingsExpo, Adam Justice, vice president and general manager of Grid Connect, will review what happens when smart devices don’t work togethe...
"There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York Ci...
The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
Discover top technologies and tools all under one roof at April 24–28, 2017, at the Westin San Diego in San Diego, CA. Explore the Mobile Dev + Test and IoT Dev + Test Expo and enjoy all of these unique opportunities: The latest solutions, technologies, and tools in mobile or IoT software development and testing. Meet one-on-one with representatives from some of today's most innovative organizations
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in Embedded and IoT solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 7-9, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and E...
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web communications world. The 6th WebRTC Summit continues our tradition of delivering the latest and greatest presentations within the world of WebRTC. Topics include voice calling, video chat, P2P file sharing, and use cases that have already leveraged the power and convenience of WebRTC.