Not a Great Look (Premium)

This past weekend, Microsoft issued a one-two punch to its critics. First, it had the audacity to blame EU antitrust regulators for forcing it to change Windows in such a way that it weakened its security and was the real culprit in the CrowdStrike outage. And then Frank Shaw, a senior Microsoft executive, lashed out at reporters for not doing their jobs: If we had just known the history, he argued, perhaps we wouldn’t have all blamed Microsoft for the problems.

Frank’s comment read like a challenge. But despite being heavily invested in Windows for my entire adult life, I came up short trying to remember the event that Microsoft was referring to. And so I dove into my archives to see what I had written on this topic.

There was an obvious starting point. According to The Wall Street Journal report that contained this finger pointing, Microsoft had been forced by the European Commission (EC) in 2009 to open up Windows to third-party security companies, giving them the same level of access to Windows that it gets itself.

But I couldn’t find anything like that in my extensive archives. And so I broadened the search.

Then I landed it. In the October 17, 2006 issue of the Windows IT Pro UPDATE, I discussed some late-breaking changes to Windows Vista just ahead of its release to address the concerns of antitrust regulators in the EU and South Korea: It made its XPS (XML Paper Specification) an international standard to appease Adobe (which had correctly accused Microsoft of copying PDF), provided a search provider verification screen in Internet Explorer 7 to appease Google (which had complained about Windows Live Search being the default search in that browser), and made a security-related change that had to be what the WSJ was referencing.

“To address security company concerns, Microsoft is making three changes to Windows Vista,” I wrote. “It will provide APIs so that [security] companies can interact with the Patch Guard (or Kernel Patch Protection) feature in x64 versions of Windows. It will provide a second set of APIs so that these companies can bypass Windows Security Center, but only when they offer a competing dashboard that duplicates all the notification features from Security Center. And finally, Microsoft will advertise third-party security products in the Windows Welcome Center that appears when users first boot into Windows. This will give these companies equal access to customers that Microsoft provides for its own products and services.”

Taken together, those three security-related changes make sense in the context of an antitrust complaint: Microsoft, which dominated personal computing and was a convicted monopolist, was putting many more security controls into Windows, which obviously benefits consumers, but it was also competing with third-party companies whose businesses were negatively impacted by the changes. This is an age-old problem for platform makers like Microsoft; when Apple does this, it’s accused of “Sherlocking” app developers.

Only one of those three changes noted above is pertinent to this conversation: Microsoft had announced a feature called Patch Guard (really, Kernel Patch Protection) that was unique to the 64-bit product editions in Windows Vista. And here we get into another classic issues for platform makers like Microsoft: As technology improves and the underlying hardware platform evolves, some new features only work with the newest hardware. At that time, almost literally 100 percent of Windows PCs were running 32-bit versions of Windows, but 64-bits—x64, as opposed to x86—was clearly the future. And though Microsoft must have had some public-facing logic for locking third-party security vendors out of Kernel Patch Protection, these companies naturally saw it as anti-competitive. They saw a future in which they were needed less and less.

So what was Kernel Patch Protection?

In the January 2007 issue of Windows IT Pro Magazine, I wrote that Patch Guard—really, Kernel Patch Protection—was an esoteric security feature that had only come to the public’s attention because security firms had complained about it, knowing that antitrust regulators would be receptive.

“Kernel Patch Protection is widely misunderstood and has certainly been misrepresented to the public by the security companies,” I explained. “Kernel Patch Protection prevents what has become a common practice during Windows XP’s lifetime: Both malicious hackers and security firms have relied on the ability to patch (or ‘hook’) the Windows kernel at runtime. This ability can lead to massive system instability, since the Windows kernel is the core component of the Windows operating system, and the part of the OS that all other OS components and running applications and services require. A wide range of malicious software relies on kernel patching to infiltrate Windows, but the most-common, perhaps, is the so-called rootkit. This type of malware is often impossible to remove because of its deep hooks into the Windows kernel.”

“Security software firms began using kernel patching techniques years ago in order to battle these new, more malicious forms of malware,” I continued. “But any kernel patch, malicious or otherwise, can render a Windows system unstable and generate a Blue Screen of Death (BSOD). According to Microsoft, BSODs occur when a kernel error is so severe that the system can’t recover. The result is a nasty crash.”

Sound familiar? This sounds exactly like what happened when CrowdStrike deployed its botch security update: The impacted PCs crashed and booted into a recovery screen.

Kernel Patch Protected had actually quietly debuted first in Windows XP Professional x64 Edition a year earlier, but because that product was so rarely used, it escaped attention. Its implementation was also quite severe: It didn’t just prevent runtime kernel patching, it completely shut down the PC to protect it from attack. That was by design: Otherwise, hackers might be able to inject malicious code into the kernel while the user was fumbling with consent dialogs.

“Companies such as McAfee and Symantec, which have built powerful businesses by protecting individuals and businesses against the electronic threats that endanger Windows systems, have complained that Kernel Patch Protection prevents them from providing the same types of protections they provided in Windows XP,” I wrote. “Microsoft counter-argued, correctly, I believe, that Kernel Patch Protection makes 64-bit Vista versions more secure and stable and rendering their kernel patching methods unnecessary and obsolete.”

Why would Microsoft bow to antitrust regulators on this matter?

Simple: It was a delicate time for the software giant, which was immersed in a decade-long set of antitrust battles, had to conform to the requirements of a 2004 EU antitrust decision. And it had struggled to turn its far-reaching “Longhorn” visions into a shipping product. Microsoft hadn’t shipped a major new version of Windows in over five years, and now regulators were threatening to block the release of this next Windows version, Vista, unless Microsoft addressed their concerns. At this point, Microsoft would do almost anything to avoid a bad outcome that would delay this product any further.

And so it capitulated.

“In the days before Windows Vista was finalized, Microsoft announced a compromise: It will create a set of APIs (application programming interfaces) that will enable security software firms to interact with Kernel Patch Protection at a programmatic level, providing them with at least some of the kernel patching functionality they’ve requested,” I wrote at the time. “Microsoft says it will deliver these APIs in late 2007, perhaps as part of Windows Vista Service Pack 1 (SP1), which is due at the same time as Windows Server ‘Longhorn’. This timetable has generated a second round of complaints from the security firms, which argue that the wait is too long. However, x64 uptake won’t pick up in the first year of Windows Vista’s availability. While it’s likely that most Vista users will move to x64-based systems in the future, that transition will take years. In the meantime, users of Vista x64 PCs will be safer with Kernel Patch Protection in place.”

With that in mind, consider again how The Wall Street Journal sort-of quoted Microsoft’s statement about this event. (It’s not a direct quote.)

“A Microsoft spokesman said it cannot legally wall off its operating system in the same way Apple does because of an understanding it reached with the European Commission following a complaint,” the publication explained. “In 2009, Microsoft agreed it would give makers of security software the same level of access to Windows that Microsoft gets.”

Notice anything curious about this blurb?

I found a few issues.

First, Microsoft said that it “reached an understanding” with EU regulators that prevents it from locking down Windows similarly to how Apple does with macOS; this is a reference to an earlier part of this article that notes that “in 2020, Apple told developers that its macOS operating system would no longer grant them kernel-level access. That change was a pain for Apple’s partners, but it also meant that a blue screen-style problem couldn’t happen on Macs, said Patrick Wardle, the chief executive of Mac security maker DoubleYou.”

This is clearly a reference to Kernel Patch Protection. Which Microsoft first implemented in 2005 (in XP x64 Edition), 15 years before Apple did. But more to the point, it was “an understanding.” The EU didn’t force Microsoft to change Windows. It asked Microsoft to address the complaints it had raised, and this was only one of them. This change to Kernel Patch Protection was of Microsoft’s design, and it (and the other changes it made) apparently worked: The European Commission never tried to block the release of Windows Vista.

But think back to my early 2007 article (which was written in late 2006 thanks to the lead time required for print magazines). It doesn’t say that Microsoft gave third-party security software developers “the same level of access to Windows that Microsoft gets.” It says that Microsoft would give them “at least some of the kernel patching functionality they’ve requested.” Some. But not all of it. If Microsoft restricted its own access to live kernel patching—and it did—then that, too, was a product design decision that it made.

To be clear, these decisions were grounded in a desire to make Windows more secure. But the resulting changes were not designed by regulators. This was Microsoft’s work. And so the implication that Windows would have been more secure if it hadn’t been for those meddling antitrust regulators is a bit disingenuous. Microsoft could have pressed its case, could have demonstrated that this kernel access by it or third parties was both “unnecessary and obsolete.” But it decided that getting Vista out the door was the most important concern.

There’s more.

It says that Microsoft “agreed” to make these changes in 2009. These changes, as noted, were announced in 2006, and they were implemented in Windows Vista Service Pack 1 (SP1) when that update shipped in 2007, as originally promised.

“Service Pack 1 includes supported APIs by which third-party security and malicious software detection applications can work alongside Kernel Patch Protection on 64-bit versions of Windows Vista,” Microsoft’s release notes explained at the time. “These APIs have been designed to help security and non-security ISVs develop software that extends the functionality of the Windows kernel on 64-bit systems, in a documented and supported manner, and without disabling or weakening the protection offered by Kernel Patch Protection.”

This seems to refute the Microsoft sort-of quote in the WSJ over the weekend (along with that 2009) date. At that time, Microsoft said the changes it made to address the concerns of the European Commission did not weaken Kernel Patch Protection. Today, it claims otherwise. Poor Microsoft, a victim of its success.

More to the point, the European Commission never asked Microsoft to weaken Windows security to prop up competitors. (Note that McAfee and Symantec were, and still are, U.S.-based companies, just like Microsoft.) It was simply asking for Microsoft, which was then operating under the shadow of its antitrust punishments, to play fairly.

But as I think back on this time, a time which I, again, had largely forgotten, another thought occurs. And this is perhaps a good time to bring back the incredible quote from Microsoft chief communications office Frank Shaw, who lashed out at news reporters for not understanding the history of this work over the weekend.

“A Microsoft spokesperson would not have to make this point [that the changes that enabled the CrowdStrike outages came out of an agreement with EU regulators] if the reporters did their jobs,” he tweeted.

This rubbed me the wrong way the second I read it, for obvious reasons. And because I couldn’t remember the history, which surprised me. But we need to put this in the context of what’s happening today in 2024. The European Commission is immersed in a record number of investigations against Big Tech firms as I write this, and today, Microsoft is a relatively small player in personal computing compared to bigger abusers like Apple. Windows, once the most dominant product in personal computing, is a distant third behind the ecosystems that Apple and Google created and maintain. Indeed, it’s largely an afterthought: Where Microsoft bundling Internet Explorer with Windows was at the heart of the company’s first epic antitrust trials in the U.S. and Europe, its current web browser, Edge, is such a non-event that it’s not even considered a gatekeeper under the EU Digital Markets Act (DMA).

Only two of Microsoft’s consumer-facing products, Windows and (for some reason) LinkedIn, qualify for DMA oversight. Microsoft has changed Windows 11 to comply with the DMA, but it has also dramatically weakened the default apps interface that arose out of, wait for it, its original U.S. antitrust case, and with no ill effects.

Why on earth would Microsoft still allow third parties to patch the Windows kernel at runtime in 2024? This isn’t an antitrust concern, if it ever was, it’s a security issue.

Unless it isn’t. In other words, this most recent ability to live patch the Windows kernel has been in place for 15 years. It’s either safe or it’s not. If it’s safe, then Microsoft’s allegation in the WSJ is an exaggeration, if not a lie. If it’s not safe, then this is ultimately Microsoft’s responsibility. It could have changed the product to be safer, as it apparently intended to do in 2006 with the x64 versions of Windows Vista, at almost any time in the past several years. But it didn’t.

Either way, it’s not a good look. No matter what you think about Microsoft and security.