Next Up for the Windows Insider Program: Changing the World

Posted on November 29, 2016 by Paul Thurrott in Windows 10 with 42 Comments

Next Up for the Windows Insider Program: Changing the World

Dona joins Windows Insiders for a Create-a-thon event at the New York City Microsoft Store.

Microsoft’s decision to test Windows 10 in the open was historic. But it has even bigger plans for the Windows Insider Program, which has grown into a true community of people who now have the power to change the world.

You may have heard some of this story before, as Microsoft publicly discussed the history of the Windows Insider Program at Build 2016. But in speaking with key members of the team over the past few months, I feel like a more complete picture has emerged. And that includes new details about the origins of the Insider program as well as what Microsoft has planned for the future.

So let’s start at the beginning.

The Windows Insider Program—like Windows 10 itself—has its roots in the past. And while I often like to characterize what’s now called “Windows as a service” as a reaction to mobile, the truth is a bit more nuanced. In reality, it grew out of frustrations with the way that Microsoft had been developing Windows prior to Windows 10.

Windows 7 and 8, for example, were developed over traditional three-year product cycles. The development of each was marked by just a handful of public beta releases, and while Microsoft professed a desire for feedback from each, it was withholding a secret truth: By the time the first pre-release drop had headed out to customers, it was already too late for them to provide feedback that could impact the product in meaningful ways. That is, Microsoft’s monolithic approach to creating Windows simply wasn’t working.

“We were working on Windows 7 for approximately 800 days before we shipped the first preview release to customers,” Microsoft director of program management Bill Karagounis told me.

Microsoft corporate vice president Gabe Aul concurred, noting that two-thirds of the work Microsoft would undergo on a new Windows release was complete before they shipped a public preview. “By the time we got feedback, it was past the point where we could make substantial changes and still ship it on time. That traditional beta program started feeling outdated.”

The three-year release cycle finally started to unravel with Windows 8.1, which shipped in just 18 months but was still developed using Microsoft’s traditional development process. It was a baby step, but an important one, as Microsoft was responding to customer feedback from Windows 8, mostly in the form of complaints. But the team responsible for building Windows knew they needed to do more.

The catalyst for the change we now call the Windows Insider Program, or WIP, was Terry Myerson. Mr. Myerson had run Microsoft’s mobile business since 2008, and led the drive for Windows phone. But in July 2013, he was promoted to executive vice president and given control of Windows and all of Microsoft’s client operating systems.

As you may recall, Windows 8.1 shipped in October 2013. That was under Terry’s watch, so to speak, though of course the feature-set was completed before he assumed control. A subsequent release which I feel was just as important—Windows 8.1.1, or the Windows 8.1 Update—shipped in April 2014, answering further criticisms of Windows 8 and paving the way for Windows 10.

But by that point, the seeds had been planted, and Windows development was about to change dramatically. Terry and his team were talking about what would become Windows as a service. So Aul, Karagounis, and others saw this the natural time for a similar overhaul to the customer feedback loop.

“Our engineering team had been moving the product forward,” Karagounis told me. “But we wanted to get users’ voices into the cycle as early as possible.”

According to Mr. Karagounis, a small team from Windows Fundamentals pitched the notion of what became the Windows Insider Program in January 2014. The idea was as simple as it was radical: Build Windows 10 in the open, with the public. Release new pre-release builds much more frequently than in the past. Get feedback from customers who would help design the product from an early point. And then keep iterating, so that Windows 10 would always be “evergreen,” as both Mr. Aul and Karagounis described it.

Mr. Myerson loved the idea. And he green-lighted it immediately.

In April 2014, the Fundamentals team presented a plan for what it then called the Windows Preview Program. The goal at the time was to ship the first preview build in October 2014. To accomplish this, the team had to scale its build flighting capabilities far beyond the needs of the internal employees who had previously been the target.

“Our original goal was to allow up to 400,000 people to participate in the program,” Karagounis told me.

That sounds laughable today, given that there are millions of people in the Windows Insider Program. But here again, credit must be given to Terry Myerson, who reviewed the plans two weeks before Microsoft went live in late September. He had two key pieces of feedback.

First, the name sucked.

“I thought you were going to build a community,” Mr. Aul remembers Terry challenging him. “This doesn’t sound like a community.” Terry kept coming back to the term inside—as in, “these users need to feel like they work inside Microsoft, are a part of team”—and suddenly the name was obvious. The Windows Preview Program was renamed to the Windows Insider Program.

Terry wasn’t done. Told that the team was targeting 400,000 Insiders, he blanched.

“No,” he told them flatly. “Everybody can be a Windows Insider.”

Scrambling because its systems couldn’t handle more than 400,000 users, the Fundamentals team was able to turn things around pretty quickly. Which worked out well as the Insider program passed the 400,000 user mark within two or three days, Aul told me.

“It picked up and leveled off,” he said. “It took us about a month to get to 3 million people.”

Some of the people behind WIP: Gabe Aul, Dona Sarkar, and Bill Karagounis.

Some of the people behind WIP: Gabe Aul, Dona Sarkar, and Bill Karagounis. Oh, and Ninja Cat.

I was also happy to discover that Microsoft had on its own come to the same conclusion that I had, that the company was perceived as huge and faceless, an enormous corporation. That had to change, the team decided, and it realized that its communications with Insiders had to be more personal.

“In September 2014, Joe Belfiore and I were discussing how we should position WIP,” Mr. Aul told me. “We knew we would have a website and an email message welcoming people. But who signs the email? Joe and I argued that it had to be a person, that we needed a human connection. I can’t come from the Windows Insider Program; that would be too anonymous. The question was, who?”

Microsoft’s PR team told them that they didn’t need to worry about that: PR had people trained in outreach and communications.

“That didn’t sit right with me,” Aul confided. “The people who signed up for WIP would be very technical. They would want answers fast, and we’d be on Twitter. Joe and I were discussing options, and we had a few people in mind.”

But then Joe said, “Gabe has been around longer than anyone in Windows.”

And that’s true: Gabe Aul has been at Microsoft for 24 years. He started in product support and then moved into testing after about three years.

“I spent three years taking calls from customers,” he told me. “As a result, I have a real passion for paying attention to what customers care about and are saying.” And most of his career at Microsoft has been about feedback: He’s spent many, many years improving Windows based on customer feedback.

Sounds like a perfect fit, right? There’s just one problem: Aul is an introvert.

“When I started communicating about the Windows Insider Program,” he said, “I had 8 followers on Twitter. 6 were bots, and I’m pretty sure one of them was my mother.” Today, Aul has over 130,000 followers. He is, shall we say, a bit more well-known outside of Microsoft’s build labs.

“It was a weird moment of celebrity,” he conceded.

I recall meeting Mr. Aul for the first time at the October 2014 event at which Microsoft announced Windows 10 and the Windows Insider Program. He’s instantly likable, which won’t surprise anyone who has followed him on Twitter, especially during his turn as the face of the Insider Program. But how the heck had I never even heard of the guy before? I’ve been writing about Microsoft—and Windows—for over 20 years myself.

As it turns out, it’s because Aul had always worked his magic from inside of Microsoft. He was the wizard behind the curtain.

“I had always worked to improve the products we build based on the real world,” he told me. “But it’s one thing to test in the lab, where we run millions of tests each day, but under controlled conditions. The goal here was to have the craziness of the real world enter the loop. That’s what I’ve been working on [more recently].”

With the Windows Insider Program successfully launched in late 2014, Microsoft set out to constantly iterate, essentially applying the principles of Windows as a service to its own testing process. And that meant evolving the feedback system based on, yes, feedback.

The first change was this notion of Fast and Slow rings.

“When we started, WIP had a sort of one size fits all approach,” Karagounis said, “with one set of users, and we gave them builds on a slower cadence than today. But we learned that some people want to be on the edge, and get builds as fast as possible. Others want a more measured thing, with builds coming maybe just once a month. Then there are those who want to live on the released OS, but experience the latest app updates. So we made those changes over time.”

The result is more builds, more quickly, for those that want them. Some weeks, we even see two releases to the Fast ring.

“The Windows Insider Program has been such a great learning experience,” Aul told me. “It’s made us revisit what ‘fast’ really means. When you’re coming off a three-year plan, 30 days seems really fast. But some participants were telling us they wanted builds even faster.”

To enable this change, Microsoft flipped the build ring structure so that Insiders on the Fast ring now get builds even faster than do internal employees. This makes sense on a number of levels, but the most obvious is the make-up of the respective audiences. Sure, some employees are willing to keep testing new builds, but most just need to get work done. Insiders, meanwhile, have explicitly joined the program to test Windows and impact new features and functionality. So this change is obvious in retrospect, but it wasn’t foreseeable in late 2014.

As a result of all those changes, Microsoft can now turn around a build and flight it to Fast ring testers in just four days. You’ll recall that it took Microsoft about 800 days to ship the first Windows 7 preview. “We’re really proud of this change,” Karagounis told me.

“We’re not building an echo chamber,” Terry Myerson told me. “People want to be part of the creative process, and we want them there.”

Looking ahead, there are more changes coming to Windows 10 and the Windows Insider Program, and Karagounis hinted that the recent news about the Unified Update Platform (UUP) technologies was an “important piece, a major milestone.” Microsoft is also doing work to allow IT pros to leverage WIP so that they can better understand and deploy Windows 10 in their organizations and streamline the process so that their users can authenticate to the program more easily. “It’s exciting in terms of where we are going,” he said.

It is exciting. But there is perhaps no more exciting change to the Windows Insider Program than the arrival this year of Dona Sarkar, who has almost single-handily led a charge to evolve WIP into a community that can do good around the globe. I’ve spoken to Dona several times in recent months, in places like Toronto and Boston, and for our more recent conversation, she was Skype-ing in from Nigeria, where a rolling blackout briefly halted things.

Peaceful transition of power: The ceremonial hand-off of the Big Red Button.

A peaceful but playful transition of power: The ceremonial hand-off of the Big Red Button.

Dona isn’t the person many of think of when we think about Microsoft. And that, arguably, is what she’s trying to change.

Like Aul, Karagounis, and Myerson, Ms. Sarkar has been at Microsoft for many years, in her case since 2005, and she joined the Windows team immediately upon her arrival. She’s worked on such Windows technologies as AutoPlay and Windows Search, and perhaps most tellingly, she was part of a team that worked to get early feedback reporting built into Windows.

More recently, however, Dona led developer engagement for Microsoft’s HoloLens project. Meaning that she was essentially doing the Insider Program, but for HoloLens.

Paving the way for Dona’s latest roll was a shift in Gabe Aul’s responsibilities. In July 2015, Mr. Aul became corporate vice president for Engineering Systems for Windows and Devices, essentially moving back into the “engine room,” as he calls it. For several months, then, he did two jobs, as Microsoft has never worked out a plan to change WIP leadership.

“Joe [Belfiore] and I had talked about this at length,” he told me. “The job should rotate, we thought, and shouldn’t be one person forever. But we had no plan for making that happen.”

Eventually, and predictably, the demands of the two jobs proved too much for Aul. So the team began looking around for a replacement.

“I had worked with Dona before on Windows 8 and earlier,” Aul told. “She’s super-impressive. But she wasn’t actively looking to make this change, but once she heard about the opportunity, she agreed to taking over WIP in June 2016.”

If you’re not familiar, Dona Sarkar is a dynamo—in the words of Wikipedia, she is “an aspiring fashion designer, a fashion blogger, a speaker and an author of four published books, which include two novels, a novella and a career advice book”—and her ability to bring people together and get things done is immediately impressive. So it’s perhaps not surprising that she quickly set out to expand on the community core of the Insider Program.

Dona has spent much of the past year engaging with Insiders, both virtually and in-person. And based on what I know about her travel schedule—New Zealand and Nigeria, the latter twice, most recently—she doesn’t have a lot of down time. Among her many recent activities are various #WINsiders4Good “Create-A-Thons” at which Insiders get together to solve local problems, a #WINsiders4Good fellowship in Lagos, Nigeria that has yielded dramatic results, and a partnership with Code.org for a global Windows Insider Hour of Code effort.

Dona demonstrating HoloLens at a Co-Creation Hub Nigeria event.

Dona demonstrating HoloLens at a Co-Creation Hub Nigeria event.

But one of her more powerful ideas is that the WIP community doesn’t need to be solely virtual. There are, she says, Insiders in almost every country on earth, people who are creators who can help one another. They just need to find each other.

“We have people with skills,” she says. “All we need to do is make the connections. This is so powerful. Maybe you’re an app developer who needs a graphics designer: Well, we have Insiders who do that. It’s like the world’s biggest business school. You are never alone: There’s always another Insider near you.”

To make this dream a reality, Microsoft will over the next several months revamp the WIP website and will create location-based forums so that Insiders who opt in can find each other. “I call it the ‘Debug the world’ database,” she said. “It’s very practical as a business school.” It will start small, with social profile pages for Insiders and grow from there, she said.

I see this evolution as a nifty 21st-century approach to user groups, which have fallen by the wayside with the rise of the Internet. But web searches and other digital interactions lack a key element that user group meetings still provide: That in-person interaction which so often can lead to meaningful decisions and actions. “It’s about making connections between people,” Dona told me.

For the Windows enthusiasts in the audience, the changes brought about by the Windows Insider Program are truly amazing. Aul estimated that there were 1-1.2 million downloads of Windows 8 across its handful of preview releases, but that the Insider program today is “exponentially” bigger.

“There are millions of Insiders today, and to my knowledge, WIP is the largest public software preview program in the world, certainly the largest of its type,” he told me. “It’s significantly bigger than anything Microsoft had ever done, and the number of downloads for Windows 10 preview releases is now hovering around the 100 million mark.”

But the changes coming beyond Windows are perhaps even more impressive.

“The Windows Insider Program is not about pushing your favorite features,” Dona told me. “It’s about achieving something and impacting everyone on the planet. This message resonates with people who are in WIP: It’s like the 5-5-5 plan: We have five days to fix bugs, five months for new features, and five years for innovative new technologies. For the next five years, we’re really talking about the next five billion people.”

And that’s what expanding the Windows Insider Program is all about, she says.

“Insiders are the millions who represent the needs of the billions.”

Premium Comments (31) Comments (11)
Elevate the Conversation!

Join Thurrott Premium to enjoy our Premium comments.

Premium member comments on news posts will feature an elevated status that increases their visibility. This tab would allow you to participate in Premium comments with other premium members. Register to join the other Premium members in elevating the conversation!

Register or Subscribe
31 Comments
Sort by Votes | Date
  1. 9 | Reply
    MattHewitt Alpha Member #110 - 1 week ago

    Awesome article. After finishing reading it I had to scroll back up to the top to check whether or not it was premium. Isn't flagged that way, but it was so informative it could've been.

    1. Paul Thurrott
      9 | Reply
      Paul Thurrott Alpha Member #1 - 1 week ago
      In reply to MattHewitt:

      Yep. I felt like this one should be for everyone. 

    2. 6 | Reply
      winlonghorn - 1 week ago
      In reply to Paul Thurrott:

      Excellent article! This is one of the reasons why I am proud to support your work! I always enjoy reading what you have to say on a wide range of subjects!

    3. Paul Thurrott
      0 | Reply
      Paul Thurrott Alpha Member #1 - 1 week ago
      In reply to winlonghorn:

      Appreciated, thanks.

    4. 0 | Reply
  2. 2 | Reply
    davidD Alpha Member #2138 - 1 week ago

    Great, informative article. WIP has really gone very far in such a (relative) short time - imagine where it will be in 2 years time!

    Well done Paul, this could easily be a Premium article! (Well done on you for not making it one though)

  3. 1 | Reply
    rtodd_us Alpha Member #1012 - 1 week ago

    I really enjoyed this article.  Thanks!

  4. 0 | Reply
    glenn8878 Alpha Member #2387 - 1 week ago

    Windows 10 is a two for one OS, but they didn't test sufficiently for tablets and mobile. The tablet experience is not up to par. When will they get around to it?  It's been 4 years since Windows 8 debut and the tablet experience regressed.  They lack a sound strategy. They can't compete in mobile. Luckily, desktops is what matters, but this gravy train won't last forever.

  5. 0 | Reply
    AliMaggs Alpha Member #1317 - 1 week ago

    Amazing article. I love your opinion pieces (which is why I subscribed to Premium) but this piece is outstanding. Great combination of your inside knowledge, interviews/quotes, and forward thinking.

  6. 0 | Reply
    ChristopherCollins Alpha Member #2122 - 1 week ago

    This was a truly great article, Paul.  Nice pics too.  I try to keep a PC on the program.  I play it safe and stay in the release preview ring on my personal machine.

  7. 0 | Reply
    fanchettes - 1 week ago

    Excellent article! I hadn't heard of the #winsiders4good movement before. Glad the community is about more than a laundry list of complaints.

  8. 0 | Reply
    cseafous Alpha Member #610 - 1 week ago

    Great article Paul.  It almost makes me want to buy an extra Windows device so I can join in.  I wonder if the feedback they receive is applied in other ways.  For example, do they use the telemetry they receive to build the hardware they use for internal testing?  It seems like that would be a good way to determine if the issues that surface affect everyone or just those with certain hardware configurations.

  9. 0 | Reply
    jimchamplin - 6 days ago

    Wonderful article, Paul. Its stuff like this that's the reason I'm a subscriber. Thanks!

  10. 0 | Reply
    tgazzano Alpha Member #2072 - 6 days ago

    Great Article Paul, thank you.

  11. 0 | Reply
    John Dunagan Alpha Member #1483 - 6 days ago

    I'm all for Windows Insider Program changing the world, but at the risk of sounding doom and gloom, can we please first observe some movement on even one or two of the unaddressed show-stopper, BSOD-based issues that are in the current Fast Ring build? We haven't had one since 14971, and the deal was that in return for understanding stuff would be broke (like the webcam), that we'd be getting new builds on the quick to at least move the issues along. I admire all the Microsoft personalities, too, but there's still a job to do.

  12. 0 | Reply
    Vuppe Alpha Member #1076 - 1 week ago

    Ah, Paul. This is why I subscribed to Premium. Articles like this. Great info on a great program. Thanks. Back when I had a Windows phone I was always on the Fast ring. I don't have a spare desktop to set up on the Insider Program, though, unfortunately.

    (Yes, I know it's not a "premium only" and that's fine. I don't have premium so I can get nice things, I do it so everyone can have nice things...and so I don't see ads.)

  13. 0 | Reply
    mmcpher Alpha Member #245 - 1 week ago

    Next up for W.I.P.?  More Builds please, like yesterday!  I need a new PC Fast Ring to fix my keyboard/trackpad on my SP3!  I signed on from the start and have rarely suffered because of it, as I've been able to work with most builds (on PC and WM).  W.I.P. has gotten so big it doesn't quite seem like you're connected with other Insiders.  When everyone in the world is connected, no one is "connected".  What's my feedback worth, when there's millions always plugging away?  But the WIP team has evolved and the feedback is now aggregated in a way that allows you to quickly upvote formerly obscure requests or bug reports.  I appreciate the outreach and also the listening aspect.  I've been in rooms with developers who seemed genuinely interested in how end-users use their products, but I get the feeling when they get back to the office where its "just us geeks", they quickly drift back to developing apps for themselves instead of end-users.  With WIP, Microsoft tries to convey in its posts accompanying new Builds, a little bit of how they incorporate feedback and telemetry.  It helps to be collectively heard even if you are not individually acknowledged.   I also like the Garage Program.  Maybe Microsoft can bring some WIP into that also?  

  14. 0 | Reply
    chrisrut Alpha Member #193 - 1 week ago

    Thanks Paul - one of the most interesting articles in ages.

  15. 0 | Reply
    Onaropus Alpha Member #739 - 1 week ago

    Great article Paul, your commitment to quality shows.  

  16. 0 | Reply
    the_zeni - 1 week ago

    Great article, thanks Paul!

    I really like the idea of bringing together People with the help of WIP. Sounds like a great Addition to the already running Winsiders4good project.

     

    But to me there is a downside of the WIP I'm surprised you didn't mention: greatly reduced internal testing (together with the rumored layoffs of the testers). While it is obvious to me that you don't test Builds that flight to Insiders, the ones going to the current branch really need to be tested in a similar fashion they were with previous versions of Windows.

    Today this seems not to be the case. And together with the somewhat insufficient "Feedback Hub" tool (where niche/locally restricted cases mostly get ignored) leads to bugs getting into the public release cycle that never should and may even damage the Windows reputation.

    1. -1 | Reply
      Siv Alpha Member #451 - 1 week ago
      In reply to the_zeni:

      I was thinking exactly that, it's all well and good having these Windows insider programs (I am one albeit not active until we get nearer to the next major release) but I think it's not working.  If it was we would not have seen the debacle with:

      1.  Webcam issue that Paul suffered from.

      2.  The freezing issue.

      3.  Machines that upgraded from previous Windows versions and users finding they can't save into their own documents and pictures folders.

      4.  Outlook stops sending after the upgrade to Windows 10.

      5.  All the Surface issues with CPU, hot bag, batteries.

      6.  Edge comes out with no "save as" on downloads, doesn't offer to import settings from IE and still doesn't offer right-click "refresh" screen option!

      7.  Start menu stops working so user can't exit Windows unless they know about the right-click menu!?

      I could go on, but all of this is very simple stuff that should never have got past the 3 million Windows Insiders. Someone must have reported it! So that tells me that either MS just never looks at Insider bug reports unless 10,000 people reported it.

      Alternatively maybe the Insiders are a specific bunch who actually know how to do an sfc /scannow and fix the issue themselves and so it doesn't get reported at all and this just won't help "normals" who don't know what to do.

      I am beginning to think they should refine a proportion of the Insiders to be a much smaller number like say 100,000 and make sure that it is made up of many different types of users with as diverse a range of hardware as possible and of differing levels of ability and include some "normals" who don't know/want to tinker and fix things themselves like real users do! Then use them as a "tester" ring and really concentrate on that bunch and look at every issue reported rather than being swamped with the 3 million early adopters/enthusiast who could be used for bug reporting but probably more to get ideas about new features and improvements to the UI.

      I think they also should get in some more testers at MS as some of the above issues should have been picked up by proper testing without needing any Windows Insiders.

      For normal users who need their Windows PCs for real productive work, the bugs that have escaped particularly in v1607 are unacceptable. It does make me wonder if there is anyone left at Microsoft who cares about people who sit at Desktop PCs and do real work rather than playing with toy town UWP apps that do nothing of any great consequence.

      I deal with small business clients who run accounts packages, large databases, complex linked spreadsheets, CAD, Development and general business applications who have had at best a sub optimal experience with Windows 10. Most are complaining about the constant issues after updates and long down times when going from the initial release to 1511 to 1607 and frankly after 1607 I am dreading Redstone. These failings are definitely hurting Windows' reputation and a lot of businesses are looking at Apple or Linux if this keeps up!

  17. 0 | Reply
    daveevad Alpha Member #1614 - 1 week ago

    Excellent! What a great article. Thank you.

    This may even have earned you a few travel days without a build being flighted...worth every word ; )

  18. -13 | Reply
    MutualCore - 1 week ago

    This encapsulates Microsoft. Fire all the testers and make the rest of us test for them. Genius.

    1. Paul Thurrott
      5 | Reply
      Paul Thurrott Alpha Member #1 - 1 week ago
    2. 1 | Reply
      johnlavey Alpha Member #860 - 1 week ago
      In reply to Paul Thurrott:  I am retired now but have always been excited about computers.  The Windows Insider Program continues to interest me.  
      I had to scroll up at the end of the article to double-check the author's name.  I was surprised...and not surprised.  I thought it might have been written by a Microsoft employee. Paul, YOU, are the ultimate Windows Insider because you most often manage to bring to me (and all your readers, I hope) exciting information about computers, and particulary Microsoft. This article ties in the history and exciting changes that the WIP encompases in a very informative and enjoyable manner.  Thank you.      

       

    3. 4 | Reply
      jrswarr Alpha Member #114 - 1 week ago
      In reply to MutualCore:

      You don't get it - this puts us right in the mix. It's almost like being part of the development team. Is this good for MS? You bet - but I think it's even better for us - because we get to steer MS to the OS we really want.

    4. 3 | Reply
      Bart Alpha Member #117 - 1 week ago
      In reply to MutualCore:

      You don't get it

    5. 2 | Reply
      nightmare99 Alpha Member #1133 - 1 week ago
      In reply to MutualCore:

      Hey at least you can put on your CV that you were involved with the development of Windows 10 ;)

    6. 1 | Reply
      winlonghorn - 1 week ago
      In reply to nightmare99:

      Good point! Never thought of that one! lol

    7. 0 | Reply
      AliMaggs Alpha Member #1317 - 1 week ago
      In reply to MutualCore:

      This is the first Premium comment I've ever down-voted. Well done. Achievement unlocked.

    8. 0 | Reply
      jimchamplin - 1 week ago
      In reply to MutualCore:

      Tone-deaf response.