From the Editor’s Desk: The Problem is You (Premium)

In October 2022, Linus Torvalds merged the first Rust code into the Linux kernel. It’s been a firestorm of controversy ever since. And that controversy just claimed its latest victim, Asahi Linux lead Hector Martin, who has stepped down from that project and will no longer contribute Linux kernel code because of separate run-ins with kernel maintainer Christoph Hellwig and Torvalds himself.

From the outside, it may seem odd that OS kernel development could be, in any way, controversial. But Torvalds, for all his contributions to the industry, has always been a bit controversial, and the system he’s put in place to keep the kernel moving forward mirror his personality in the same way that other industry icons like Bill Gates and Steve Jobs influenced their respective organizations.

Torvalds’ involvement in this episode is negative and unfortunate. But it’s also beside the point. The real story here isn’t found in specific rebukes or hurt feelings. This is really about a problem our industry has faced since its inception. It’s about the awkward generational transitions that occur when older, more experienced technical people suddenly stop evolving with the times, become change averse, and push back against younger, for forward-leaning colleagues. It is, in other words, about psychology, not technology.

But we have to understand the technology to understand this story.

As you may know, Rust is a C-like, memory-safe programming language that’s been embraced by systems programmers for its unique blend of safety, performance, and capabilities. It’s so good, and so popular, that it’s triggered a long-overdue transition away from the popular but unsafe and bug-prone C programming language. It’s especially ideal for OS kernel and driver development that Torvalds adopted it for Linux kernel development in 2022. Microsoft followed in 2023, with Azure CTO Mark Russinovich declaring an end to new C and C++ development. Going forward, all new kernel code for Windows and Azure should be written in Rust, he said.

This seems straightforward. Rust delivers all the benefits of C with none of the downsides, so there’s no compromise. Clearly, we can move on to a safer, Rust-based future and collectively reap the benefits. I imagine a future in which OS developers use AI to rewrite code blocks one-by-one in Rust in a perfect act of code refactoring with literally no downsides.

There’s just one problem, and I’m looking at you, graybeard. You know who I mean. The perhaps overly confident middle-aged and older senior developers who in this case stand between us and that safer future. In a pattern that replays itself everywhere in technology and at all times, these older developers, who grew up on C and now understand it just as well as they understand their native spoken languages, are resistant to change. And they’ve been fighting against the addition of Rust to their beloved codebases since this shift began.

Aside from the Torvalds angle, we hear about this with Linux development because it’s done in the open. Microsoft may have embraced open source, and I do believe that it loves Linux now. But its core OS development across Windows, Windows Server, and Azure is as proprietary as ever. And so it protects this secret sauce like a modern day Colonel Harland Sanders, with whatever discussions or arguments it may have internally occurring behind closed doors. (Many closed doors. I imagine Microsoft’s kernel development happening in a secret underground lair similar to that seen in the opening credits of Get Smart.)

To be clear, this isn’t just about software development. It’s about the Exchange administrator I’ve discussed so many times, forced to hand over the reigns to his kingdom to Microsoft 365 in his final act in that role. It’s about the Mac-using wife of a friend who craps all over the bugs in Windows despite having not used it in over 7 years. It’s about the AI deniers, who tried it once, discovered that it just produced garbage, and then never tried it again, instead spending their time crapping on this technology and trying to exorcize it from the products that are all racing to adopt it. It’s about people who entering an industry that is all about change, embracing that change, and then making careers for themselves and calcifying into a state in which change, suddenly, is the enemy.

This isn’t naive. No, not all change is good, and not all proposed changes are the right way forward. This is not about blindly accepting change, it’s about blinding rejecting change. It’s about being incapable of even evaluating change fairly. And then not trying regardless. Worse, actively fighting it.

My Exchange admin story is representative. Here was this guy on the descending side of the bell curve of his career who had come into this job bright-eyed and eager to do the right thing. He had presided over two or three Exchange migrations, during which he skipped every other version–just being prudent–and got pretty good at that one major task. During this, he stopped learning about anything unrelated to his job, he was too busy, and the work was stable. And then he stopped learning about anything new that was related to his job, too. And by the time the cloud arrived and the infrastructure he was managing on behalf of his company had become bloated, expensive, and complex, he was incapable of doing the right thing. This person was never going to make the right decision for his employer. By this point, he was more concerned about himself.

That’s not selfish, it’s just human nature. The ability to change isn’t necessarily inherent in all of us. But it is something we can unlearn, much in the way a muscle will atrophy if it’s not regularly exercised. The day to day happens, there are bad times and good times, and it’s suddenly several years later and the world has changed. And we didn’t see it coming. Our younger selves would have ridden this new wave, but our current, older selves choose to resist. We want to stop the wave.

The Linux kernel flare-up, something I’ve been following but maybe wouldn’t normally write about, is likewise representative. It stars Linus Torvalds, who created Linux, for God’s sake, and still maintains the Linux kernel over 30 years later. And it stars a cast of characters no one outside that tiny, insular world, myself included, knows anything about. I will forget their names the second I finish writing this article. But the names are not really important. What’s really happening here is that there are old people and young people. The old people are resisting change. And the young people are pushing change.

They are both wrong. Not all the time. But each has, in their own way, has made mistakes. And neither side seems particularly willing to admit that or, more importantly, trying to work on their own shortcomings.

Here’s what happened.

Torvalds, as noted, committed the first Rust code into the Linux 6.1 kernel in October 2022. He followed that up with the first Rust-based driver code in December 2022. Most of the subsequent major Linux kernel releases–6.2 in February 2023, 6.3 in April 2023, 6.4 in June 2023, and so on, has included more Rust code.

This doesn’t sit well with some people. In late 2024, a Microsoft software engineer who ran Rust for Linux, stepped down from the project, citing vague “nontechnical nonsense” for sapping the “energy and enthusiasm” he once had for the project. He linked to a specific part of a video demonstrating the disconnect between those who understand and thus prefer C and those who want Rust to save the day, noting “no one is trying force anyone else to learn Rust nor prevent refactorings of C code.”

“I truly believe the future of kernels is with memory-safe languages,” he wrote. “I am no visionary but if Linux doesn’t internalize this, I’m afraid some other kernel will do to it what it did to Unix.”

That anyone would debate this fact seems absurd. But as The Register pointed out at the time, a comment to that video reads, “You’re not going to force all of us to learn Rust.” It’s impossible for me to read that and not think of the AI deniers who clog my social media feeds and this site’s comments complaining about Microsoft adding Copilot to everything. They’re framing an objective improvement, a new feature, as something being forced on them by some authoritarian overseer.

Here’s the other side of this coin.

In the programming language world few of us know or care about–this is, admittedly, a dry topic–there have been many attempts to improve on C’s limitations and problems over the years. Some were high-profile extensions to C, most notably C++ and Objective-C. But others have come as improvements to the C language itself. And some Rust deniers would like to see the improvements that Rust brings simply be added to C. Why learn a new language when C can simply be improved?

That makes sense to me. There are parallels here with the story Terry Myerson told me after he announced Windows 10 for Arm in collaboration with Qualcomm. “I just want to get all those f#$king Intel stickers off the laptops,” he said. Point being, Intel needed competition, Qualcomm and Arm offered some notable advantages over x86, and maybe this competition would result in Intel improving x86 accordingly. In other words, it didn’t matter if Qualcomm “won,” it would be just as good for Intel to catch up in the areas that Arm was superior.

The problem, of course, is that much of the resistance to Rust is just resistance to change, full stop. It’s not fair to judge anyone on a single utterance, but the video comment noted above betrays a lot about the line of thinking. No one is forcing the entire group of Linux kernel maintainers to learn Rust. But Torvalds did evaluate the language and found it to offer enough advantages over C (or assembly language) to allow its use in the kernel. And he went on to accept multiple Rust code submissions into the kernel. Rust is part of the Linux kernel and its use will only expand if it proves its worth. It appears to be doing so.

In the most recent episode, Hector Martin, who created Asahi Linux to bring Linux to Apple Silicon-based Macs, submitted what’s described as a proposed “DMA allocator abstraction” that would allow drivers written in Rust to access the C-based Linux kernel’s memory access APIs. And kernel maintainer Christoph Hellwig didn’t just reject the proposal, he poured gasoline on it and set it on fire.

“No Rust code in kernel/dma, please,” he wrote in the Linux kernel mailing list, referring to those memory access APIs. (Via The Register, again.) When someone objected, he continued with what can only be described as an unprofessional and enlightening response.

“Don’t force me to deal with your shiny language of the day,” he wrote. “Maintaining multi-language projects is a pain I have no interest in dealing with. If you want to use something that’s not C, be that assembly or Rust, you write to C interfaces and deal with the impedance mismatch yourself as far as I’m concerned.”

Getting past the inflammatory stuff, here I see both sides of that coin again. That “not in my backyard” argument made by someone apparently uninterested in being “forced to” learn Rust. And then a potentially reasonable technical argument for where Rust/C interop should occur. I don’t know enough about this topic to determine whether he’s correct. But the person who pushed Hellwig does. And he said that Rust code that abstracts the C code in the kernel is maintained by Rust developers and only impacts drivers written in Rust. And that this system would be better than requiring each Rust driver handle the bindings individually. Fair enough.

Except to Hellwig.

“If you want to make Linux impossible to maintain due to a cross-language codebase, do that in your driver so that you have to do it instead of spreading this cancer to core subsystems. (where this cancer explicitly is a cross-language codebase and not Rust itself, just to escape the flameware brigade),” he retorted. Yes, he used the term cancer … to describe Rust. “Every additional bit that another language creeps in drastically reduces the maintainability of the kernel as an integrated project … You might not like my answer, but I will do everything I can do to stop this. This is NOT because I hate Rust. While not my favorite language it’s definitely one of the best new ones and I encourage people to use it for new projects where it fits. I do not want it anywhere near a huge C code base that I need to maintain.”

This takes the argument into a new direction, and if you’ll excuse yet another complaint about the community I am very much a part of, this is something else I see all the time. A technical person pretending to compliment a technology–product, service, whatever–by stating that it’s wonderful. For other people. But not for him. I mean, he would never use it. But you go right ahead.

This is the definition of condescending, and it muddles the fair evaluation of the technical merits of both sides of this debate. Martin believes Hellwig violated some Code of Conduct rules, and he called on Torvalds to intervene. He also opined that the Rust coders should simply ignore Hellwig, submit the code, and let Torvalds decide that way. If Torvalds accepted the submission into the kernel, they would effectively end-run around Hellwig and his obstructionist ways.

Well, Torvalds did jump into the debate. Just not in the way Martin had hoped.

“How about you accept the fact that maybe the problem is you,” Torvalds wrote in the Linux kernel mailing list, addressing Martin. “You think you know better. But the current process works. It has problems, but problems are a fact of life. There is no perfect. However, I will say that the social media brigading just makes me not want to have anything at all to do with your approach. Because if we have issues in the kernel development model, then social media sure as hell isn’t the solution. The same way it sure as hell wasn’t the solution to politics. Technical patches and discussions matter. Social media brigading – no.”

So there you go.

Torvalds has the final say in this matter. He approved the use of Rust and has accepted several Rust code submissions to the kernel. And yet, he sees in Martin’s behavior the same issues I pointed out about Hellwig. And he speaks to the divide between blind acceptance of change (in Martin) and blind resistance of change (in Hellwig). And in this case, he is saying that, having evaluated both arguments, he will simply allow the system that is in place to continue to work as-is.

I may or may not agree with that. But it parallels something I write about all the time, and live day-to-day. I am constantly re-evaluating the software, services, and products I use, including the ones that work well. (Notion is a notable and recent example.) When I make a change, it’s incredible. But it’s just as incredible for an offering to withstand the challenge. If I keep using what I was previously using, that’s fine. I know that it’s the right choice, and my opinion is up-to-date. This mirrors the Terry Myerson thing with Intel and Windows on Arm. It doesn’t matter which side wins. As long as there’s a net improvement.

Fighting change is an easy trap to fall into, all you have to do is age. But it’s wrongheaded, and this is why I sometimes have to force myself to be open-minded. Blindly embracing change is immature and silly, but finding the maturity to see past biases and at least be open to change is something that should come with age, too. It often doesn’t.

We all make knee-jerk reactions. We’re all wrong sometimes. The trick is admitting when you’re wrong. And then finding that balance between change because it’s better, and change for change’s sake. I can’t say that this issue with the Linux kernel is resolved in any way. Indeed, since Torvalds jumped in what I think of as an insensitive response, Martin left Asahi Linux too. He’s had it with the whole mess, and I don’t blame him. Perhaps Torvalds has become the Exchange admin in this story. Or maybe he’s right. I don’t know.

What I do know is that this kind of thing has been happening since the dawn of personal technology. And it’s never going to stop. We’ll be debating these types of issues until we’ve all gotten so old that we just don’t care anymore. And when that happens, a younger generation will step in and make changes. Until they, too, inevitably age and repeat the same pattern. Again and again and again.