So you know your Beatles from your Beach Boys, do you? And Elvis from Elvis Costello? Do you pontificate on the fine differences between Bebop and Hard Bop? Tired of DJs that don't know anything about the music they are playing and who never play the songs you want to hear?

Imagine having songs - from any number of music genres - being chosen and played for you by a system that understands your personal musical preferences! What if you could automatically have a play list created that cuts across all musical genres but still matches that funky groove you've been into lately? It would be like having your very own personal, intelligent DJ on hand, twenty-four hours a day, seven days a week! MusicMagic" from Predixis® knows your musical preferences and tailors play lists to suit your tastes perfectly.

You might not think that a complex, automated preference matching system like MusicMagic could be built using the Linux operating system. Yet that is exactly what Predixis® did. Linux offers a superb operating platform for MusicMagic. The convenience, low cost, stability, reliability, and performance of Linux allow MusicMagic to be constantly available, 24 hours a day, 7 days a week, 365 days a year. Trouble-free, always on - this is crucial to the continued viability of MusicMagic, and Linux makes it happen.

Linux. . . Then Mistrust
Many in the software community have long doubted the ability of Linux to offer comprehensive, reliable services for commercial business applications, including Web Services. While the reasons for this lack of confidence are complex, the bottom line has really been a general uneasiness that consumers and businesses have had over Linux as an open source operating system. Continual updates, no central development authority, lack of unified technical support, and a wide variety of competing Linux "flavors" have also been seen as liabilities by the business world at large.

Much of the negative publicity over Linux has come directly from purveyors of traditional, windows-based operating systems, including industry giants such as Microsoft, who have been quick to exploit drawbacks, both real and imagined. In the past, Microsoft has committed itself to several well-publicized debates over the usability and efficiency of Linux, going so far as to openly criticize various Linux implementations and their creators. Under the guise that open source poses a severe security risk to the IT world at large and to software intellectual property holders in particular, Microsoft has, according to Scott Ard, indicted Linux as "a cancer that attaches itself in an intellectual property sense to everything it touches."

When it came time to put MusicMagic into development, the feud between Windows and Linux was at a fever pitch, and it was difficult to ignore. Predixis needed an operating system that could reliably run a Web services application without a large expenditure of cash or time spent in customization and implementation. The choices boiled down to three viable types of operating systems. We could go with a commercial Unix implementation, but that would have been prohibitively expensive. Microsoft offered a variety of Windows flavors but we doubted their reliability in a commercial Web services application - we literally couldn't afford to hire someone to reset a Windows-based OS each time it crashed. And finally, there was Linux, much maligned and unproven but readily available on the cheap. As a startup company, Predixis was unable to financially procure a commercial Unix operating system. Linux was inexpensive, but could it do the job? The deciding factor came down to Microsoft's smear campaign against Linux. If Microsoft was this afraid of an up-and-coming operating system, then there must be some validity to proclamations coming in from the pro-Linux developers from around the globe. Our decision was relatively easy. Cheap, unproven, and hated by Microsoft. Linux was our first and only choice.

Our rationale was recently validated by IBM's exciting announcement that it was committed to research and development for Linux applications and technology. Another validation that we had chosen the right operating system for our Web services implementation was Microsoft's continuous and ineffective attacks on Linux. As we built MusicMagic, the flexibility that Linux offered us - especially in terms of multiple architecture choices - made it clear that Microsoft's claims against Linux were not motivated by technology but by simple economics. Microsoft stands to lose a great deal of money pushing proprietary software, with hefty licensing fees, when open-source applications such as Linux can be had for free under the terms of the General Public License (GPL). Indeed, the GPL seems to be Microsoft's real target, and not Linux per se. One doubts that Microsoft and its allies would have much problem with Linux if it were for sale in the same way that Windows is for sale.

There are other start-up companies that are just now in the position Predixis was in the late 1990s, trying to decide whether to move their information systems and new development projects to a Linux-based architecture. Our story of how we integrated Linux into MusicMagic, coupled with IBM's strong and reassuring commitment to Linux, should send a resounding message that Linux has reached a level of maturity and stability needed to become a full-fledged part of any technology-based business solution. Linux in the MusicMagic Architecture

MusicMagic is an n-tier distributed application using Web services for communication between the constituent parts, which include a database, a preference-matching engine, and a user interface module. It is as modern an architecture as can be achieved using current development standards. We chose Web services - and in particular SOAP-based messaging - as the communication protocol between these modules because it allowed us to isolate the functionality of the different parts of the system using a very elegant and simple architecture. This emphasis on elegance and simplicity is very important. Since MusicMagic is cutting-edge technology, we were constantly discovering new challenges during its creation; it is not overstatement to say that MusicMagic proved to be a living project that constantly changed form during development. Because of this, it was imperative that the architecture remain as simple as possible in order to allow the different parts of the system to change and grow independent of one another. If we could not meet this requirement, then the entire system would prove too big to continually refine and develop.

Therefore, distribution and simplicity were key features of the system from the beginning of development, and SOAP allowed us to meet each of these requirements with ease. The different modules in the MusicMagic system contain different functionality, and when these modules changed or evolved, those changes became visible only through the SOAP interface, which we could control. SOAP keeps these different modules, all running on separate machines, communicating effectively, so much so that information exchange between modules never proved, or even promised to be, a bottleneck within the system. The true bottleneck to the system was the music processing requirements, used to populate the database with data (more on this bottleneck in a moment).

Because SOAP keeps the modules communicating so effectively, we were able to then turn our full attention to overall system performance and functionality, to creating an architecture that would prove to be simple, effective, and easy to use, without having to worry about how the individual pieces would communicate. Figure 1 shows the basic MusicMagic architecture that was developed. The main elements are, as previously mentioned, a database, a preference-matching engine, and a user interface module. The database (or databases, since multiple databases can be used in this system) and the preference-matching engine are both SQL compliant with JDBC or ODBC APIs. This part of the MusicMagic system runs on top of Linux. See Figure 1: Basic architecture.

The database is a relational preference database, containing information (data) extracted from pieces of analyzed music that define the actual usable contents of the database. The preference-matching engine, when given user preferences concerning certain types of music from the user interface module, can find within the database music that satisfies these stated requirements. This engine is a series of Tomcat and Apache servers running as an integrated Web services server. The user interface module is the interface between multiple users and the MusicMagic system.

The manner in which the MusicMagic system works is as straightforward as its architecture. A user gives the interface module their musical preferences (by means of a small survey), then makes a selection of an artist or song. The user's request is fed into the preference-matching engine, which in turn queries the database (see the thick arrows in Figure 1). The database returns the original song or artist request along with a playlist of similar items to the preference-matching engine. The engine in turn delivers this data to the user via the user interface module (see the narrow arrows in Figure 1). The connections between the database and the preference-matching engine, and the preference-matching engine and the user interface module are all SOAP-based messages. Linux is being used to efficiently run the heart of the entire system, the efficient tandem that is the database and preference-matching engine.

Figure 1 represents the typical data transfer for clients that are directly accessing a commercial database of music that uses MusicMagic as its user interface module, or an in-store kiosk running MusicMagic that a customer is using as a reference tool to find certain songs or artists. The users themselves can be using any type of computing or telecommunications device to connect to the user interface module, due to the versatility of the SOAP protocol. The connection between the user and the user interface module can be an in-store kiosk, an HTML browser, a Windows-based PC, a Linux-driven hand-held device, or whatever. The possibilities for the actual user device connecting to the user interface module are really limitless, since SOAP keeps the device connections simple, standardized, and readily compliant to the rest of the system.

Another type of transaction is shown in Figure 2. This is an indirect request, where a user sends inquiries to the MusicMagic server and in return receives playlists based upon user preferences. The client then uses these playlists to download or order music from various online retailers (R1, R2, R3 in the figure below). All transactions are SOAP transactions. This scenario is a good example of a client using a portable MP3 player, PalmPilot, or similar hand-held device, where the transactions are wireless, to download music from a commercial music vendor. See Figure 2: Transaction.

Linux keeps both of these types of Web services (wireless and direct connection) running efficiently since it requires very little system resources to keep the database and the Web services server active. Database inquiries can therefore be managed with great expediency and efficiency, with little delay from a system point of view. Linux also keeps these systems extremely scalable, as any number of databases, clients, or vendors can be added as demand requires.

The true bottleneck in developing the MusicMagic system was the music processing requirements and not the system architecture, communication between modules, or the management of system resources using Linux. Populating the database required developing a music processing system, using MP3 compression formats, which enabled us to develop an incredible depth and breadth of variables so that the system itself could respond intelligently to any user inquiry. The amount of variables used, and the time needed to populate the database until it was actually usable, made up most of the development time for MusicMagic.

The music processing system is shown in Figure 3. It communicates with the database using SOAP messages (and so is a completely separate entity to which changes can be made without upsetting the functionality of the rest of the system), and it runs on top of Linux in the same manner as the core of the MusicMagic engine. See Figure 3: Music processing system.

In the past several years, the computing community has seen Linux come on very strong as a viable commercial operating system. What has changed is fairly simple - none of these fears regarding Linux have come to fruition. The smudge campaigns waged by Microsoft and its allies has also heavily backfired, thanks in part to a media-savvy IT community that clearly saw through such inappropriate scare tactics.

Linux remains popular, and continues to gain popularity, in large part due to a core set of technical merits. These same technical merits are a large part of the success behind MusicMagic. Linux offers MusicMagic . . .

Small footprint. Linux doesn't require vast amounts of system resources to run, and is therefore perfectly suited to embedded systems and other implementations that have small system overheads.

This host of technical merits, and others not in this list, make Linux a powerful force in the IT community. Its emergence as a viable commercial operating system makes it perfect for Web services applications such as MusicMagic.

Where Do We Go From Here? A Cautionary Ending
Linux may be written for programmers by programmers, but its future depends upon its ability to support viable commercial applications. There is a dangerous mindset that has recently surfaced within the Linux community that such commercial applications are mere "toys" whose construction is "a trivial exercise left to the student." This amounts to little more than snobbery among the hardcore Linux faithful, who seem to have a vested interest in seeing the vast capabilities of Linux remain entrenched, and unused, within the walls of the development community. Such insularity will eventually cost Linux dearly in its race to remain competitive with OS offerings from rival software camps. Linux must continue to branch out into the commercial realm, powering other exciting applications such as MusicMagic. Only when Linux proves itself as a powerful engine within Web commerce will it have finally arrived as a true force in the IT community.

Note: MusicMagic is not currently a live site, if you would like to view it or get additional information, please contact Marc Campbell at mcampbell@predixis.com

Published Dec. 8, 2002 — Reads 9,127
Copyright © 2008 SYS-CON Media. All Rights Reserved.

About Marc Campbell
Marc Campbell is CEO of Predixis. Marc Campbell has 19 years applied experience in the advanced computing business. Marc has practical experience in all phases of product development and launch including analysis, design, rapid prototype, and production. Marc has a MSEE in intelligent computer systems from USC. Marc has worked for a variety of large and small technology companies including Parasoft, Northrop Grumman, Hughes Aircraft Company, Spectradyne, Custom Music Systems, and Argus Technologies.

About Kevin Coffee
Kevin Coffee is a senior technical author at Parasoft Corporation. He holds a Ph.D. and Masters Degree in English from Stanford University, where he taught composition and expository writing courses in addition to American Literature courses, and a Bachelors Degree from the University of California, San Diego. Before coming to Parasoft he worked as a technical writer at Intel. He has written articles on C/C++ development and error prevention for StickyMinds.com and Parasoft's quarterly newsletter Runtimes.