Linux Containers Authors: Liz McMillan, Elizabeth White, Zakia Bouachraoui, Pat Romanski, Stefana Muller

Related Topics: Linux Containers

Linux Containers: Article

Dissolving the Limits of Linux: The Breakneck Evolution Continues

Dissolving the Limits of Linux: The Breakneck Evolution Continues

In January, as thousands of Linux OS enthusiasts streamed onto the show floor and into conference sessions at the 2004 LinuxWorld Conference & Expo, they caught up with an old friend who many barely recognized.

The breakneck evolution of Linux into a viable environment for enterprise, research and development, and high-performance computing has converted skeptics who once dismissed the open source environment as a low-cost - and not particularly scalable - alternative to more robust and proven operating environments.

To many commercial users, the earliest implementations of Linux didn't seem suited to drive anything more mission-critical than an edge server, some desktop systems, or a few specialized applications. But much has changed since those early days. Committed and enthusiastic developers have worked hard to push Linux beyond its old limitations, and the results have touched the industry. Sensing a good thing and seeing the opportunity to finally have a unified common operating system across applications and hardware, users and vendors are endorsing Linux as the "one-Unix" once sought by POSIX standards. The increased attention has resulted in added momentum behind Linux and increased expertise in the community that has allowed a multiplying effect of innovation to occur. At LinuxWorld 2003 in New York, attendees saw the unveiling of new systems designed to scale well past the operating system's storied eight-processor ceiling. Silicon Graphics, for instance, debuted a new Linux server and supercluster family - dubbed SGI Altix 3000 - that scales efficiently to 64 processors under a single Linux kernel. This product introduction has been accompanied by hardware vendors who continued to increase their support for Linux by introducing a variety of platforms (both 32- and 64-bit), in addition to growing momentum among ISVs. Support from the Linux community, hardware vendors, and developers made Linux ready for the most demanding customer environments. With Linux now poised for success at the highest levels, corporate bureaucracy and stagnant IT organizations constitute the last rapidly crumbling barrier.

In the past year, developers and users have seen that Linux, with a 2.4 kernel and enhanced software and utilities, has what it takes to drive massively scalable systems to solve HPC-class problems in commercial settings as well as in campus labs. Today, in fact, Linux habitually resets industry expectations of its scalability and applicability. As Linux on HPC platforms like the SGI Altix continues to shatter records on the highest end of computing, a rapidly growing number of end users are embracing Linux as a production environment for running everything from databases to computational fluid dynamics analysis.

Converging Trends

The rise of Linux has resulted from a confluence of crucial industry trends and the dedicated efforts of the open source community, with help from hardware and software solution providers.

64-bit Processor Technology
Cost-efficient processor technology helped prompt the Linux community's initial success in HPC by enabling hobbyists to join the global collaborative movement, and by adding to and improving the operating system's functionality. The relatively recent introduction of 64-bit technologies capable of unseating RISC processors has been vital to the development of Linux as a mature, robust, and performance-oriented operating system as an alternative to proprietary Unix implementations. The wide availability of 64-bit processors has spawned a new generation of powerful servers and supercomputers based on the Intel Itanium 2 family, AMD Opteron, and IBM POWER 4. These systems comprise a growing landscape of options for users, establishing a competitive marketplace for Linux servers that simply did not exist before. This competitive Linux server market is forcing innovation...and that's good.

HPC System Architectures
While standard "white-box" solutions inevitably find their way into any server market segment, they do little to push the boundaries of the Linux operating environment. Because price/performance is always a consideration for Linux users, the challenge for hardware vendors is to build sufficiently differentiated solutions from industry standard components.

In the HPC space, hardware vendors must meet the dramatically increasing expectation for systems to accommodate ever-growing data sets and computational needs of technical and enterprise users - needs not adequately addressed by standard solutions for those who solve complex problems in oil and gas exploration, manufacturing, aerospace design, earth and planetary sciences research, and life sciences. These applications tend to generate terabyte data sets, which in turn create I/O bottlenecks and demand enormous memory availability. As a result, they require system architectures designed to leverage each component to its fullest. SGI's built-in memory channel interconnect fabric with a 3.2GB/sec aggregate bandwidth, for instance, allows the system to deliver data across nodes five to six times faster than conventional clustering interconnects. Beyond this, Altix class systems uniquely provide up to 8 terabytes of total addressable memory. With system architectures affording so much throughput and memory availability, more complex data sets and complete workflows can be driven entirely out of memory, enabling productivity breakthroughs that traditional Linux can't tackle.

It was clear early on that the ability to scale Linux is imperative to meeting the needs of cutting-edge users. System architectures that support reliable scalability - whether scaling up from 4 to 16 processors, or from 256 to 512 - are also vital to Linux evolving as a serious multidiscipline enterprise computing environment. NASA Ames Research Center recently installed an SGI Altix 3000 system that demonstrated 512 processors in a single system image - the largest Linux supercomputer of its kind ever created. By scaling in near-linear fashion to more processors than even some Linux enthusiasts thought possible, the NASA system was the first computer in history to break the 1 terabyte/second barrier on the STREAM Triad benchmark, an industry-standard measurement of memory bandwidth. By the standards of most users, NASA's needs are extreme. But its efforts to throttle Linux into performance overdrive offer a telling example of what Linux can do on the right platform. What's more, it credibly illustrates the Linux community's ability to outpace proprietary solutions, and provides a proof-point of how far Linux can penetrate industry and science in particular.

64-bit Applications
With the initial success of these systems has come growing support from commercial software suppliers, whose customers hungrily demand ever-heightening levels of system scalability. Customer adoption has fueled aggressive Linux porting and development among ISVs - some of the world's most powerful and sophisticated technical applications now run on Linux: BLAST, GAMESS, NX.Nastran, STAR-CD, and Gaussian. Meanwhile, leading database applications from Oracle, Sybase, and MySQL provide even more options for high-performance production Linux environments.

Linux has also proven itself outside of its old perceived boundaries. By expanding its reach to end users in a vast array of markets, it's increasingly clear that Linux can serve as a unifying operating environment, rather than the affordable substitute applicable only to specific systems or application areas.

Ready for Prime Time

As the attendees of this year's LinuxWorld discovered, Linux is on an evolution curve unlike any other operating system in history. The achievements of just the past year have helped shape this OS into a production-quality environment. Customers around the world depend on it for mission-critical applications, including Tata Motors, NASA, the University of Cambridge COSMOS Project, SARA (Dutch National HPC and Networking Center), Marathon Oil Co., and the Pacific Northwest National Laboratory.

Some are first-time Linux users; others, long-time devotees. But whether investigating the origins of the universe, predicting the effects of global warming, or hunting for oil beneath the North Sea, all of these users reap the benefits of each other's efforts. They are part of a global Linux community that, every day, astounds the world with its achievements.

By all accounts, 2004 will be another groundbreaking year for everyone developing, supporting, and deploying Linux. Just imagine what surprises are in store for LinuxWorld 2005.


More Stories By Jason Pettit

Jason Pettit has been working with Linux systems since 1998. As product manager for Linux and Altix 3000, Jason is leading SGI's charge to scale Linux to even greater heights.

Comments (10)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

IoT & Smart Cities Stories
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
DSR is a supplier of project management, consultancy services and IT solutions that increase effectiveness of a company's operations in the production sector. The company combines in-depth knowledge of international companies with expert knowledge utilising IT tools that support manufacturing and distribution processes. DSR ensures optimization and integration of internal processes which is necessary for companies to grow rapidly. The rapid growth is possible thanks, to specialized services an...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...