Now back to the notion of a "Patch Penguin," that is, someone dedicated to tracking all the patches submitted to Torvalds and making sure none of them get dropped inadvertantly. This ties directly to the VM problem because patches submitted by van Riel for his VM code were simply ignored by Torvalds, and these patches could have prevented many of the VM problems seen in the earlier versions of the 2.4 kernel. And in spite of a "healing" of the rift between Torvalds and Cox which has been reported in the press, there are recent signs that Cox is unhappier than ever.

Granted, you need a very thick skin to participate for long on the Linux kernel mailing list, and Alan Cox is not known for pussyfooting around controversial issues, but even in that context Cox has scared some people on the list with a few recent comments. Things like this snippet taken from a post Cox made to the list: "If you want something in 2.5 don't bug me. I simply don't care."

Torvalds has spoken in the past of the evolutionary nature of the development of Linux. For the first time -- at least the first time that I am aware of -- major figures in both the Linux kernel developer community and in the greater circle of open source/free software are openly speaking of rebellion. However, not everyone is confident that Linux can survive and thrive without its creator and lead architect at the helm.

I asked Cox how he saw Torvalds' regime being "overthrown," and whether he thought the new regime would be a simple replacement of Linus or something new altogether. He replied:

A simple replacement of Linus has the same problems the original did, coupled with probably having poorer taste and less people skills. Short term I suspect we'll have a shouting match, everyone will then go back to doing things roughly the same way but modified a bit by what they learned.

In time if Linus can't keep up something that can will form of its own accord and everyone will begin using it. egcs wasn't exactly an overthrow it was a bunch of people doing extra stuff. It won, and became gcc 3. If that was to happen with 2.5 I guess 2.5-dj would become the defacto tree, if it had happened in 2.4 2.4-ac would have become that tree.

Since then, Torvalds has announced that he is making some changes to the way he works and is testing a new tool to help him handle the patch load: a proprietary package called Bitkeeper. The first week of use, Torvalds has said, has slowed him down, which is definitely not the desired goal. But he is going to continue beating on Bitkeeper, and this bodes well for the future of Linux. Not because of the software, but because it shows Torvalds is willing to listen to the Linux developer community. A better way to handle the ever-increasing number of patches to an ever-more-complex kernel simply has to be found.

Is Torvalds now out of the woods? Hardly. I queried Bruce Perens for his thoughts on the "dropped patches" and if he thought the use of Bitkeeper was a step in the right direction. Perens replied:

Linus has been working on the kernel for 11 years now. It's time for a _long_ vacation. People burn out on projects. How could he not be burned out after this long? But I think the main criticism is directed to Linus' style of work, which was great when Linux was small. It does need to change, and source control is part of that. Just not this particular source control tool.

I think it's tragic that just as Arch goes 1.0 and is introduced to the world, Linus moves the kernel tree into Bitkeeper, but the reality is that Arch needs a bit more testing before it should host that big a project. It would be nice if they would move the work to Arch later on.

Eric (he of the parabolic voice) Raymond chimed in on the kernel mailing list as well. Raymond has lots of experience with having his patches dropped. 33 consecutive times, as a matter of fact, with his submission of documentation for configure.help. Raymond pointed out to Rob Landley, and the rest of the kernel mailing list, that "All movements founded by charismatic leaders like Linus eventually hit this same wall -- the point at which the charisma of the founder and the individual ability of the disciples he personally attracts are no longer adequate to meet the challenges of success, and some way to institutionalize and distribute the leader's role has to be found. Movements that fail to make this transition die, generally by implosion or fragmenting into feuding sub-sects."

Is Torvalds concerned about Cox's observations on the list that many of the maintainers are avoiding the 2.5 tree? Not a bit. When I asked if it were true that his development tree is being ignored by most maintainers/developers, he said:

Well, no. But you do have to realize that there are developers and there are developers. Some of the stuff going on in 2.5.x, and in particular the upheaval as a result of all the block device cleanups, means that the people who have a specific area and want _that_ code to be stable, tend to work on 2.4.x.

So in 2.5.x, there's lots of filesystem work, block device work, the USB-2.0 stuff, the new scheduler, and things like that. At the same time, people who work on the VM, for example, do not want to get bitten by unrelated changes in the 2.5.x tree, so the VM work and many of the low-level drivers are maintained largely on top of the 2.4.x tree.

Think of it this way: 2.5.x (like most early development trees) is where the upheaval happens. Things like the new device management stuff simply doesn't even _exist_ in the 2.4.x tree, and probably never will even get backported. And it will take some time before 2.5.x calms down in some areas, which means that people who don't care about the upheaval areas are definitely better off holding back.

My own rather inconsequential opinion is that things are going along pretty much as they should, but with an abnormally high fear-factor at present. I agree with Alan Cox's assessment that a new leadership would mean a downturn in "taste" and people-skills. My greatest admiration of Torvalds has always been for his ability to lead the hard-headed, highly intelligent, highly egotistical group of social outliers that make up the bulk of the community of Linux hackers from one release to the next. The problem is that it may tax even his great skills to successfully reduce the current uproar to a loud and healthy level of discontent, and continue the incredible voyage that is Linux development.

• Bitkeeper Source Management
• Torvalds, Cox Agree on the Future Kernel
• arch: A Revision Control System
• lkml