Programming Windows: Mark Russinovich Interview (Premium)

In January 2010, I visited the Microsoft campus in Redmond, Washington, and interviewed Technical Fellow Mark Russinovich about Windows 7.

As you may know, Russinovich was one of the earliest contributing editors to Windows NT Magazine, where he wrote about the Windows architecture from his perspective as a consultant and trainer who specialized in ripping into the Windows kernel. Russinovich first came onto Microsoft’s radar with his notorious revelation in the November 1996 issue that Windows NT Workstation and Windows NT Server—which Microsoft sold with different licenses and portrayed as being capable of handling different workloads—had the same code base.

“Microsoft doesn’t want you to read this article,” he wrote in that article. “At the kernel level, NT Server and NT Workstation are the same, and only a Registry key or two determines which is which. Just think about the implications.”

In 1996, Russinovich started Winternals Software, which produced systems recovery and diagnostic tools, including Winternals Administrator’s Pak, Protection Manager, Defrag Manager, and Recovery Manager. Microsoft acquired Winternals and Sysinternals (which offered free tools such as Filemon, Regmon and Process Explorer) in 2006, bringing Russinovich and business partner Bryce Cogswell on board. At the time of this interview, Russinovich was on the Windows core architecture team, which advised design teams as they brought the next versions of Windows to market. Today, he is the CTO of Microsoft Azure.

Paul Thurrott: How do you look at Windows 7 from an architectural or foundational standpoint? How did Microsoft decide what was going to make Windows Vista into Windows 7?

Mark Russinovich: Windows Vista was very ambitious in a lot of different areas. It overreached in some areas, and there were features that were miscalculated. Example: What was that feature that you could walk up to someone’s laptop with your laptop and share things?

Paul: Right, Meeting Space. It was this feature that no one understood. There was no click: “Well, it’s for peer-to-peer networking. You can go to a coffee shop.” And I thought, OK. I don’t think anyone will ever use it.

Mark: Windows 7 picked up where the Vista reset left off. We tried to be a lot more realistic about what could be done given the time frame that was set for the release. So with Vista, we were going after technology, going after features, and we’d figure out later when they all lined up into a point where we could release a product.

The Windows 7 release was: “OK, we’ve got three years, let’s figure out what can fit in those three years and try to be as realistic and accurate as possible with our predictions.” Things were mispredicted, and things did get cut along the way. But it was on a much smaller scale. There was a big emphasis on the complete end-to-end scenario, so that this technology isn’t just interesting from a technology perspective, but it’s got to fit into something useful for the customer. So Vista did take a lot of the heat for things that now, in Windows 7, are accepted.

Paul: I am with you. There’s a lot of history rewriting occurring here. Windows 7 could never have occurred without Vista, the way I look at it. So from an architectural standpoint, any major changes in Windows 7 compared to Vista on a deep level?

Mark: As far as a system-churning kind of change, nothing really. As far as system-impacting things, there are [several]. The biggest one at the lowest level of systems is Dispatcher Lock, the scheduling lock, that they got rid of. That has the biggest impact on things like server scalability. Power management: there was a big focus on that. Another thing you saw with this release versus the Vista release [was] a lot more collaboration with OEMs and hardware partners. So, for power management, there were really great interactions between us and Intel and AMD [focused on] measuring power usage and optimizing the power profiles, working on things like Core Parking, taking advantage of the new processors, Deep [Power Down] C6 states.

And speaking of collaboration with the OEMs and hardware partners, another big effort with the Windows 7 release was going to the OEMs early and helping them clean up their systems. A lot of the bad rap Windows was getting, especially with Vista, was because over time the OEMs were running out more and more stuff to try to get money off of a business that’s got decreasing margins. So there’s more and more of what’s generally called crapware on these systems. Part of that was the OEMs didn’t have the tools to know how the user was going to be impacted by these things or what to do about them. We shared a lot of our expertise. We had engineers work closely with their engineers, showed them how to use the Xperf tool in the Windows Performance Toolkit, and showed them how to measure things. We even showed specific examples of where they had their own software bundled in the system that was starting out as the machine booted, and we’d give them recommendations on how to re-architect the software so that it was out of that path since everyone measures boot time as something critical.

Paul: That’s true. I bet the big difference between a Vista PC and a Windows 7 PC in many ways—on the average PC—is in fact what you’re describing here. The PCs, or the base install of the operating system, might boot in whatever percentage. But once you start adding all that stuff, if the PC makers are working with Microsoft more, those things probably start up a lot more quickly simply because of the way they’re designed.

Mark: Yeah, our performance team looked at systems across the board, sample systems from a whole bunch of OEMs running all sorts of performance tests on them. We did this with the antivirus companies as well.

Paul: So give me your thoughts on this: one of the early debates for Windows 7 was whether it is a minor or major release, which is a semantic issue in many ways. But I look at it from a technology standpoint, from the perspective of people who have to manage and support the systems. It’s sort of a minor release because it’s the same technology, essentially—a very familiar environment. But from the end-user perspective, it’s a major release, because lots of good stuff going on in the UI. Was that even a consideration? How does Microsoft view this?

Mark: Steven Sinofsky and John DeVaughn didn’t view this release as minor or major. It’s a release. This is the cadence that we want to ship Windows by, so this is the kind of stuff we could get into the release, given this cadence. We expect that the changes they’ve made organizationally will make the system more efficient, and make it possible to get more work done in a shorter period of time. So there might be more work done in a Windows 8 time frame than in a Windows 7 time frame.

Paul: Obviously there’s a new plan in place, and I’m sure it’s a new team in many ways. So given the success of this system, it seems like this is the way it’s going to go for a while. So the plan is that Windows 8 might occur in a similar time frame. But you’re saying that because of the efficiencies, it’s possible that there might be even more of a change.

Mark: Yeah.

Paul: So from an upgrade/migration picture, one of the easy complaints about Windows 7 is it doesn’t provide for in-place upgrades from XP. What went into that decision and what are the real issues there?

Mark: Well, when you do an in-place upgrade, the test matrix for that is enormous. So, obviously, if we’re going to do an in-place upgrade, the most recent operating system is a higher priority than an older operating system that people are going to be coming from. From an enterprise perspective, it’s really not an issue because people don’t upgrade their systems, they do clean installs. From a consumer perspective, if you look at people running XP systems, they’re probably running older hardware that’s not even in the class of Vista/Windows 7 where it would make sense to do an upgrade.

In addition, if you look at trends in the past, consumers don’t upgrade either—they buy new PCs and get the new version of the operating system. So if you look at the return on investment of supporting the XP to Windows 7 upgrade path, versus the people that would actually benefit from making it easier than it is with the migration tool, it didn’t seem to make sense.

Paul: So then from a general perspective of IT pros, what are the big benefits you see for Windows 7? What are their reasons to migrate to Windows 7?

Mark: There are a few big benefits that will come when you pull in [Windows] Server 2008 R2. So there’s a big benefit, but it’s also a fairly good-sized investment to get to that benefit—things like BranchCache and Direct Access. If you look at just the Windows 7 client itself, you get a more efficient system, and the fact that end-users can do things more efficiently—they’re happier with UI changes. So there are a whole bunch of little things—the troubleshooting packs, which you can custom write and a whole bunch built in. The Resource Monitor is vastly improved over what was in Windows Vista—in fact, it seems like a lot of the Sysinternals-type functionality up to a certain point.

Paul: So looking within the context of the good/better/best kind of stuff, obviously Microsoft has the server things going on with R2 and then the MDOP [Microsoft Desktop Optimization Pack] stuff. If you could only do one, which makes the most sense?

Mark: Server, MDOP, or client?

Paul: Yeah.

Mark: Well, the MDOP people would say MDOP. I guess I didn’t even address the Server 2008 component. Virtualization delivers massive improvements there, Live Migration being the big key feature. But lots of scalability and performance improvements, and Hyper-V R2. That’s obviously a really important workload these days. The AD Recycle Bin. It’s the little things.

Paul: Yeah, it is the little things. That’s almost the message for Windows 7 when you think about it.

Mark: I think if you got a lot of little things that are nice, and don’t have any big things detracting from it—driver incompatibilities and application incompatibilities—then all those little things add up to something decent. But when you have a problem like Vista had coming out the door, it can wipe out even bigger things in terms of the value people see.

Paul: What about security?

Mark: BitLocker To Go is a big thing. And that’s again built on the foundation of stuff that was introduced in Vista. App Locker. I’m personally passionate about that whole whitelisting space because the last product Winternals made was a product called Protection Manager, which was a whitelisting product. So App Locker is a better inbox whitelisting solution than SRP [Software Restriction Policies] was previously. App Locker has some of the things that Protection Manager and some of the third-party products set up before, like being able to authorize software based on a certificate and other metadata, especially with the image like the publisher and version number.

Paul: It’s interesting in the next version of MED-V that they’re going with the previous version of Virtual PC for compatibility purposes. It doesn’t require [particular CPU support, like Windows Virtual PC and XP Mode]. Do you see virtualization having a bigger impact on the client-side going forward?

Mark: So first of all, App-V really brings you two things. One is the streaming, so being able to run software without having to pull it down and install it. And then secondly is the isolation: the isolation is something that they do underneath the application because the application model that Windows evolved with doesn’t cleanly separate application data between system settings, user data, and user settings. So that’s what App-V is doing under the hood: dynamically figuring out where those pieces of data are and separating them. That’s what the whole sequencing does, is figure out where those things are.

If we could get everybody to rewrite their apps and separate them, and then put streaming on top of that, you’d basically have App-V, or what you wanted from App-V: Being able to have applications side by side, and having the dependencies nicely-identified, their states separated so you could toss changes and go back to a good point. So the way I see App-V evolving is us trying to go in that direction with applications in general, not just relying on this trick underneath to get applications to do the right thing.

And as far as virtualization on the client, this is something that we’ve thought long and hard about, and are still thinking long and hard about, and the question is: are there any scenarios where there’s compelling value to having machine virtualization on the client that makes up for the increased management cost and performance degradation that you would get out of it? If you take any particular scenario where you say, “We could do that with machine virtualization,” then what we do is say, “Well, is there any way you could do that with Virtual PC type of virtualization, or within the Windows box, and does that make more sense?” So, what is the value that machine virtualization is bringing?

Paul: I think any form of virtualization, regardless of where you go, gives you an interesting way to cut with the past, because by providing a previous version of Windows in a VM, all of a sudden there are these old APIs that you don’t have to include now in the base system. That gets interesting.

Mark: We’d love to be able to move on to a newer, better, more coherent world, but mixing the old and new is something people are going to want to do. From a UX perspective, an application interoperability perspective, a systems management perspective, that’s where all the seams in machine virtualization show up and cause problems. You can patch over some of it, with things like the integration stuff in XP Mode where things show up in the Start menu, but it’s still not seamless from a management perspective or a UX perspective.

Paul: Apple recently described Windows 7 as old technology, which I found somewhat hypocritical given that UNIX is the basis of Mac OS X. How do you react to a comment like that? I mean, obviously, there’s old stuff.

Mark: The big value of Windows is the fact that it’s old technology that runs everyone’s apps. If we came out with an operating system that looked like Windows but couldn’t run your Windows apps, it wouldn’t be Windows. Nobody would want it.

Paul: It’d be Ubuntu.

Mark: Yeah, it’d be Ubuntu. It’d be something else. And so, the value of Windows is being able to carry things forward and improve the experience—manageability, security, reliability—along the way.