Welcome!

Linux Containers Authors: Liz McMillan, Pat Romanski, Elizabeth White, Yeshim Deniz, Jyoti Bansal

Related Topics: Linux Containers

Linux Containers: Article

Is Windows 2003 Server really faster than Linux/Samba?

Here comes Windows 2003 Server! And it's faster than Linux, because Microsoft cheated with the benchmarks!

(LinuxWorld) — Microsoft funded and released a new set of benchmarks to follow the debut of its often-renamed Windows 2003 Server. The boys in Redmond are proud as punch of their new baby, whatever its name. They would have the world believe that Windows 2003 Server is twice as fast as Linux, at least when it's used for file serving.

Microsoft hired VeriTest to conduct benchmarks using hardware and software configurations provided by Microsoft. VeriTest shares Microsoft's penchant for name changes. VeriTest used to be known as ZD Labs, and has also been known as eTesting Labs. The benchmark results VeriTest produced for Microsoft showed Windows Server 2003 to be faster than Red Hat Linux/Samba on both uniprocessor and SMP HP servers — a stunning 66-to-95 percent faster, at that. Given the many configuration options for Linux, Samba, and Windows, I had questions about the validity of VeriTest's results.

Reactions from the Samba team

I spoke to Jeremy Allison, head of the Samba team, to get his reaction to the benchmarks. Allison provided a few insights into the test configurations that don't leap out at me because they are hidden away in appendixes to the benchmark document. Allison feels the hardware configuration Microsoft chose is substantially responsible for the outcome.

Benchmarks aside, I asked Allison what he would tell someone who approached him about wanting to save money by moving from Microsoft servers to a Linux/Samba solution, but was also interested in how they compared in performance. Would he say, for example, that the performance of the two were comparable? Or that Linux was slightly slower? That it was slightly faster?

Allison minced no words about it. "I would say that actually we are faster." But then he added this caveat, "In all the earlier benchmarks, we have been. Before I would say that against (Windows Server) 2003, I would want to have the chance to repeat this benchmark. I'm hoping that we do get the chance."

If you look through the various benchmark results listed on Dan Kegel's NT-vs.-Linux benchmarks page (the URL is provided in Resources below), you'll see what Allison means. In September of last year, Red Hat Linux Advanced Server 2.1 bested Windows 2000 Advanced Server in both performance and price/performance in a TPC/C comparison.

In June of 2001, Ziff-Davis/eWeek compared Web-serving on Windows 2000/IIS and Red Hat/Apache, Red Hat Apache with TUX, and Red Hat/TUX. Windows 2000 just barely nosed out Red Hat/Apache, but against TUX, Red Hat's Web server for SMP machines, it was hopelessly outclassed. TUX scaled to more than twice as many clients as the Windows machine.

For those benchmarks, by the way, technicians from Dell, Microsoft, and Red Hat were present and assisted in setting up and tuning the systems. That's definitely not the case in the VeriTest benchmarks. As Allison put it, referring to the most-infamous benchmarks of the past, "It's Mindcraft all over again."

RAID 0, Windows 1

Allison pointed out the heart of the problem in the most-recent benchmarks immediately, though it was hidden away in Appendix B of the 21-page document.

"Note that they used RAID 0, not RAID 5," he said. "Linux RAID 0 performance isn't terribly good for software RAID; (it's) much better with software RAID 5. On Windows, performance with RAID zero is very good."

(Editor's note: The PDF file states VeriTrust used hardware RAID on both systems.)

He strengthened that argument by pointing out the throughput-performance graphs that are part of the benchmark results. "If you look at the curves in the benchmarks, what is really really obvious is that Samba, or the kernel, isn't running out of steam," Allison said. "What's running out of steam is the disk subsystem."

The graph below, taken from the VeriTest benchmark, illustrates the problem.

Throughput comparison

RAID 0 wasn't the only poor choice for Linux in the tests. "They used ext3, which is one of the slowest filesystems on Linux," Allison said. "In a real comparative test, you would use XFS, you would stripe it appropriately, doing exactly the same thing for the Windows system, of course. Then you would put the journal log on a separate device."

Allison believes those two changes alone — changing from RAID 0 to RAID 5 and from ext3 to XFS — would end the I/O bottleneck that flattened Linux's performance-curve very early in the VeriTest benchmarks.

More tweaks

Allison's analysis does not end there. He also pointed out ways in which the Windows system had been tweaked to make it perform at its best. As the benchmark document clearly states on page 7: "For the File server performance testing using Windows Server 2003, we performed a series of operating system and testbed client tunings as specified by documentation provided from Microsoft."

Part of the tuning that Microsoft provided VeriTest, for example, was a registry setting that turns off the standard Windows 8.3 file-naming convention. Another tweak was made to TCP stack on the client machines. Yet another tweak was made to the buffer-cache pool on the server. Obviously, Microsoft invested considerable time and effort in finding the best possible configuration. On the Linux side of the table, VeriTest relied only upon their own in-house expertise, bolstered by a couple of books and Internet searches.

Allison concluded our brief visit by stating, "I'm not saying that we, the Samba team, could have produced the same benchmark going in the opposite direction." Allison clearly wants to do his own benchmarks to straighten the record. Maybe Red Hat or another Linux vendor will step up and provide the hardware and software required to do just that.

More Stories By Joe Barr

Joe Barr is a freelance journalist covering Linux, open source and network security. His 'Version Control' column has been a regular feature of Linux.SYS-CON.com since its inception. As far as we know, he is the only living journalist whose works have appeared both in phrack, the legendary underground zine, and IBM Personal Systems Magazine.

Comments (4) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Jean Marc 06/01/04 05:25:31 PM EDT

When I bumped into the Linux trashing pages of Micro$ofts website, I got slightly unwell in disgust... Sure Windowze performs 200% better than Linux, especially in assimilating virusses and worms in their operatED systems, but in about all other areas I have my doubts about "better performance".

My god what a bunch of #$%-holes they are. Again they are doing what they are brilliant at: selling lies, for that's what all their Marketing is about. How many more bullsh#$ will we have to take from the Redmond boys???? I just wish there was a law against that kind of practises.

JM.

Arthur Bennett 09/19/03 12:48:46 AM EDT

Hi
Just finish reading the interview with the Sambab man Mr Allison.
Is Windows 2003 Server really faster than Linux/Samba?

Very interesting testing procedures by MS and the company they used to benchmark their system. Its a case of "I'm the biggest and I can do what ever I like. Its unreal how this just continues. If you were thinking about purchasing OS to do this particular job, you would think Windows would be the best choice by their benchmark results. It could be if the test were more balanced but they weren't. So the general public gets done again by big corporations.

Well I hope we will see a more balance testing review on Windows 2003 server comparing it with Samba.

regards
Arthur
Linux Student
Perth WA

Arthur Bennett 09/19/03 12:48:38 AM EDT

Hi
Just finish reading the interview with the Sambab man Mr Allison.
Is Windows 2003 Server really faster than Linux/Samba?

Very interesting testing procedures by MS and the company they used to benchmark their system. Its a case of "I'm the biggest and I can do what ever I like. Its unreal how this just continues. If you were thinking about purchasing OS to do this particular job, you would think Windows would be the best choice by their benchmark results. It could be if the test were more balanced but they weren't. So the general public gets done again by big corporations.

Well I hope we will see a more balance testing review on Windows 2003 server comparing it with Samba.

regards
Arthur
Linux Student
Perth WA

Jim Sherhart 08/04/03 05:14:03 PM EDT

Joe,

I would like to point out that Snap Appliance recently completed performance benchmarks of the Linux-based Snap Server 4500 against three Microsoft Windows Powered OS (SAK) units with a different outcome. We did test in RAID 5 and our Snap Servers do use XFS, so I think these results parallel many of the comments made by Mr. Allison. If you happen to look at the test results available at Veritest.com, you'll notice two things: (1) there is very little difference in terms of hardware and (2) no special configurations of any kind was performed on the test equipment. I realize that SAK 2.0 is not the same as Windows 2003, but I thought the results would be of interest to your readers.

Regards,

Jim Sherhart
Product Manager
Snap Appliance Inc.

@ThingsExpo Stories
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @ThingsExpo Silicon Valley Call for Papers is now open.
SYS-CON Events announced today that DivvyCloud will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. DivvyCloud software enables organizations to achieve their cloud computing goals by simplifying and automating security, compliance and cost optimization of public and private cloud infrastructure. Using DivvyCloud, customers can leverage programmatic Bots to identify and remediate common cloud problems in rea...
SYS-CON Events announced today that Tintri, Inc, a leading provider of enterprise cloud infrastructure, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Tintri offers an enterprise cloud platform built with public cloud-like web services and RESTful APIs. Organizations use Tintri all-flash storage with scale-out and automation as a foundation for their own clouds – to build agile development environments...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry’s single source for the cloud. Fusion’s advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus intern...
SYS-CON Events announced today that Progress, a global leader in application development, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Enterprises today are rapidly adopting the cloud, while continuing to retain business-critical/sensitive data inside the firewall. This is creating two separate data silos – one inside the firewall and the other outside the firewall. Cloud ISVs ofte...
SYS-CON Events announced today that Systena America will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Systena Group has been in business for various software development and verification in Japan, US, ASEAN, and China by utilizing the knowledge we gained from all types of device development for various industries including smartphones (Android/iOS), wireless communication, security technology and IoT serv...
DevOps at Cloud Expo – being held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real r...
SYS-CON Events announced today that Hitachi Data Systems, a wholly owned subsidiary of Hitachi LTD., will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City. Hitachi Data Systems (HDS) will be featuring the Hitachi Content Platform (HCP) portfolio. This is the industry’s only offering that allows organizations to bring together object storage, file sync and share, cloud storage gateways, and sophisticated search and...
The 21st International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Machine Learning and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding busin...
SYS-CON Events announced today that Technologic Systems Inc., an embedded systems solutions company, will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Technologic Systems is an embedded systems company with headquarters in Fountain Hills, Arizona. They have been in business for 32 years, helping more than 8,000 OEM customers and building over a hundred COTS products that have never been discontinued. Technologic Systems’ pr...
SYS-CON Events announced today that Carbonite will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Carbonite protects your entire IT footprint with the right level of protection for each workload, ensuring lower costs and dependable solutions with DoubleTake and Evault.
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet th...
SYS-CON Events announced today that Progress, a global leader in application development, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Enterprises today are rapidly adopting the cloud, while continuing to retain business-critical/sensitive data inside the firewall. This is creating two separate data silos – one inside the firewall and the other outside the firewall. Cloud ISVs oft...
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value S...
SYS-CON Events announced today that Cloudistics, an on-premises cloud computing company, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloudistics delivers a complete public cloud experience with composable on-premises infrastructures to medium and large enterprises. Its software-defined technology natively converges network, storage, compute, virtualization, and management into a ...
SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists will look at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deli...
We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.