I recently held a focus group and reaffirmed what I’ve known for years: Microsoft’s Signature program delivers the clean and reliable PC experiences that its customers deserve, and should demand.
Disclosure: I was commissioned by Microsoft to organize a focus group to help determine whether customers would prefer a PC configured with or without Signature. I also prepared a written report for Microsoft that summarizes the results of the study and then presented my findings and recommendations to Microsoft privately. But my opinions about Signature are my own, and I have no qualms—moral or otherwise—recommending this program to readers of this site. —Paul
You may recall that this is the second time I’ve performed a focus group study for Microsoft Signature. This study differed from the original 2011 study in a number of ways—back then, Windows 7 was the current Microsoft OS, for example, and the size of this more recent focus group was over twice as big as the original—but the results were very similar: PCs configured with Signature were overwhelming preferred by a very diverse set of customers over otherwise identical stock PC configurations. And in my own testing of a subset of focus group test PCs, the results were likewise clear: Microsoft Signature is the ideal configuration for new PC purchases.
This shouldn’t be surprising to anyone reading Thurrott.com. As you know from my deliberately provocative editorial, Sorry, Satya: No One Will Ever Love Windows Until You Fix This Problem, and the ongoing Clean PC series that followed, the PC industry is at a crossroads. And PC makers are trading short term profits derived from crapware bundling for long term customer satisfaction and loyalty. Meanwhile, PC competitors of all kinds—Macs at the high end, Chromebooks at the low-end, and Android and iOS mobile devices of all kinds—are chipping away at the PC as well. The net result is that this market has been shrinking for several years, with no clear end in sight.
To its credit, Microsoft has been working on solutions, and Windows 10 in particular has helped to reverse some of the exodus that occurred in the wake of the poorly-received Windows 8. But Windows 10 introduces some interesting issues of its own, as I’ll explain in a bit. And it is with Microsoft Signature where the true hope for this industry lies.
What is Microsoft Signature?
Microsoft sells specially-configured PCs, made by major PC makers such as HP, Dell, Lenovo, Acer, and others, under its Signature brand at its physical and online Microsoft Stores. These PCs are configured from the factory with a cleaner software image than the one provided and sold by the PC maker. Microsoft removes all of the unnecessary crapware that PC makers bundle with their PCs, ensures that only the most highly optimized drivers are installed, and then delivers machines that boot, shutdown, sleep, resume and generally perform faster and continue doing so over time. And it does so at no additional cost to the customer: This is literally about making PCs better for their users.
Signature arose out of efforts to improve the PC experience within Microsoft during the Windows Vista time frame. The first result of these efforts was a program called Velocity, by which PC makers would work directly with Microsoft to ensure that the software and driver loadout on select PCs matched what Microsoft knew to be the ideal configuration with regards to performance and reliability. Some PC makers, notably Lenovo, advertised their participation in this program.
That success led Microsoft to expand into what we now know as Signature as well as related efforts such as Microsoft’s retail stores and the associated support chain. There have also been renewed efforts in the Windows 10 time frame to begin working with PC makers directly again on cleaner, more reliable PCs. (See The HP Spectre x360 is What Happens When a PC Maker Collaborates with Microsoft for an explanation of this kind of work.)
Here’s the problem. Even when a PC maker works closely with Microsoft on certain PC models or configurations, they still can’t resist bundling certain apps I describe as crapware. This includes that most lucrative of crapware, third party anti-virus and anti-malware, which is both unnecessary and overly-aggressive in its attempts to frighten users into paying regularly for updates. And even though a technical user can wipe out a modern PC and clean install Windows 10 themselves, the resulting PC build is not as reliable over the long-term as is one that comes configured with Signature from the Microsoft Store. Signature PCs are configured by Microsoft to work cleanly and ideally out of the box, of course, but that’s easy. They’re also configured to continue working properly over the long-term.
At the time of my original Signature study, I had expected PC makers to embrace this program. That Signature PCs would be offered via PC maker web sites, and in retail stores everywhere, instead of (basically) only through the Microsoft Store. That Signature would become, over time, the stock PC configuration everywhere. The benefits were—and still are—so obvious, and could result in a complete rethinking of the PC buying experience.
That has not happened, of course. And I was curious whether this second study might help shine some light on why, while establishing whether Signature still made such a dramatic difference.
The second Signature study
So in early December 2015, over two long days, I gathered over 80 people in hour-long sessions of up to 8 people each. They each compared two identical Windows 10 PCs, side-by-side, one of which was configured with Microsoft Signature, the other which was configured by the PC maker. These participants worked through a series of tasks using a worksheet in which they observed and recorded various aspects of PC performance and usage. And each participant was asked to comment on and rate each criteria, and ultimately asked to choose which PC they preferred overall. I then duplicated, and expanded on, these tests on my own, using 4 of the 8 sets of PC configurations that were used by the study participants.
The first task was the Out of Box Experience, or OOBE. During the first Signature study in 2011, we actually had each participant step through this process individually, but given how time consuming this can be, we instead showed them a time lapse video of a PC that came in at the middle of pack, from a drama standpoint: The Signature configuration was able to complete OOBE in 5 minutes and 48 seconds, compared to about 10 minutes for the stock PC configuration. Not surprisingly, 96 percent of participants who completed this portion of the worksheet stated that they preferred the Signature configuration, and as unsurprisingly, very few had relevant comments about the differences. But in my testing of the four sets of PCs later, I found that I was able, on average, to complete the OOBE on the Signature PCs about 40 percent faster than on the stock PC configurations.
Next, study participants were asked to observe the startup and shutdown times of the PCs, and again rate which one they preferred. Here, 81 percent of the participants who rated the PCs in this category preferred the Signature PC over the stock PC, with some participants noting that the non-Signature PC felt “sluggish” and less responsive.
Since some participants only measured the startup and shutdown times once—which might have skewed the results because of one-time activity anomalies—I measured them multiple times on the four PCs I tested and then averaged the results. The Signature PCs always booted more quickly, and by generally dramatic differences. But shutdown times were very close, and were a draw overall, with Signature PCs outperforming their stock cousins two out of four times.
My guess here is that ongoing optimization work in Windows over the years has floated all boats. But I will also point out that measuring sleep and resume times is probably more relevant in today’s personal computing world. Such a thing is of course beyond the capabilities of the mainstream users who participated in the study, especially given how Windows 10 segregates OS and application processes during these events.
For the third task, participants were asked to make observations about the Windows desktop, taskbar, and system tray on both PC configurations and to explain their preference. Though most didn’t fully realize this, the Signature PCs did not include any additional onscreen items beyond a stock Windows 10 install, and 90 percent of them preferred this configuration. Descriptions such as “clean,” “uncluttered,” “less complicated” and “more organized” appeared repeatedly in the worksheets they filled out.
The fourth task focused on the Start menu and produced similar results, with 87 percent of participants preferring the Signature configuration. Here, however, we saw some interesting commentary on the cluttered nature of Start menu on both the Signature and stock PC configurations. The culprit, of course, was Windows 10, which comes with an inexplicable collection of non-necessary apps.
There are “a lot of unnecessary things going on,” one tester noted … of the Signature PC. Several users in particular called out a version of the game “Candy Crush” as being particularly unwanted and unnecessary, but there are several such apps included with Windows 10 for some reason. (This is not an area where the Signature team can help, of course. Like the PC makers, they are beholden to what the Windows team places in the product.)
Focus group participants were then asked to judge the anti-virus and anti-malware solution bundled with each PC. Signature PCs come with Windows Defender, Microsoft’s free security solution, while each stock PC configuration came with time-limited trial versions of third party security software. Again, 87 percent of participants preferred the Signature PC. And again, some interesting issues were raised … about Windows Defender.
There were two notably negative reactions to Microsoft’s security solution.
First, the software is so quiet and unobtrusive that some testers weren’t even sure the PC was protected. The third party software on the non-Signature PCs was annoying in that it typically popped up immediately to warn them that the trial period was about to end. But this at least informed them that the PC was protected.
Second, some of the more technical participants explicitly distrusted Windows Defender, with one vocal dissenter noting that in his role as an IT pro, he was required to augment Defender with a third-party alternative. His workplace “does not trust Windows Defender,” he said.
Both of these complaints are understandable, though it’s not clear what the Signature team can do directly. Windows Defender is indeed too quiet, and could benefit from the “green badge of health” notification icon that Microsoft previously used on both its client- and server-based security products. As for not trusting Defender, this product has actually scored very highly in recent AV-TEST rankings. But third party security suites are of course available for those who feel they need more protection. My advice is simple: I trust and use Defender, but I do occasionally scan my PCs with a free third-party anti-malware solution just in case.
Overall, 93 percent of the participants in the study preferred the Signature PC configuration to the stock PC configuration. The Signature PCs generally provided “a better experience,” as one participant noted, and were “less cluttered and intrusive” than the non-Signature equivalent. “This is what I want a PC to be like when I buy it,” one of the more technical participants noted of the Signature PC configuration. By comparison, the same participant wrote that the OEM configuration had a “horrific user experience” with “crapware … that offered little of value.”
As a long time fan and customer of Signature, none of these accolades surprised me. But I was a bit surprised that the score wasn’t even higher. Speaking with the participants during the study and reviewing their worksheets in the weeks afterwards, I tried to better understand the 7 percent who actually preferred the non-Signature PCs. But questions remain.
The dissenting participants often noted little difference between the two PC configurations in some tests. And in many tasks, they actually rated the Signature PC ahead of the stock PC, but still rated the latter as superior overall … for some reason. We had stressed the importance of arriving at a conclusion even if the participants had not completed each task, and I wonder now if a “no preference” option might have impacted the results. I also examined the worksheets to see whether I could tell if some testers had perhaps mistakenly chosen the wrong PC, but I found no evidence of that.
One of the many commonalities between the two Signature PC studies I’ve now done is that some small percentage of users actually prefers more “stuff”—what I’d just call “crapware”—on a new PC. They see this as some kind of value, as in “more is more.” But whatever the reason—the diversity of the Windows user base, I guess—some people just don’t want a clean PC.
My recommendations to Microsoft
Given the test results and my own experiences testing four sets of PCs afterwards, I came up with some recommendations for Microsoft.
Better communication and education. The most disappointing result of this study was that, five years after the first study, so few non-technical users are still unaware that the Signature program even exists. “Consumers overall remain largely unaware of Signature and of the advantages of buying PCs directly from Microsoft,” I told them. As a result, communication and user education remain Signature’s main actionable weakness, as they were five years ago.” You can visit a Microsoft Store to see the problem, too: Surface devices are marketed as Surface. But Signature PCs sit under signs that just say “Laptops.” More can be done here.
Diversity and time to market. Like other retailers, Microsoft offers only a subset of the PC model choices that consumers see online, and of course physical store locations are rare in some areas. Worse, Signature PCs cannot be configured online as with other PCs, limiting consumer choice.
Engage with PC makers. I had expected PC makers and retailers to provide Signature PC configurations directly to customers by this point, but that hasn’t happened for a variety of reasons that are well-known to Microsoft. But in the aftermath of high profile crapware-related scandals, now is the time for Microsoft to require more from the PC makers that license its Windows products and services. How that happens—outright subsidies, whatever—is unclear. But Microsoft needs to wean PC makers off their crapware addictions.
Engage across Microsoft. Signature PCs should evolve from being clean in a Spartan sense to offering more of a “best of Microsoft” experience that can include obvious purchase incentives such as free Office 365 subscriptions. But Microsoft’s Surface lineup should be branded as Signature PCs, so that customers who cannot afford these aspirational devices will understand they’re getting a similar experience while (usually) paying less for Signature PCs. This kind of marketing would also help Surface, since it would position Microsoft’s in-house PCs “above” other devices, even those that are sold in a Signature configuration.
I can’t stress this enough: Signature matters. In a world in which the competition—again, Macs, Chromebooks, and mobile devices of all kinds—continues to get better and often provides a much cleaner experience than what customers see on Windows PCs, the time to act is now. Satya Nadella publicly stated that he wants the world to love Windows 10. But as I wrote last January, that cannot happen unless every PC sold provides this Signature experience. PC makers are ruining their own products, and Windows, by bundling crapware on their PCs.
If you’re reading this site, chances are you already know this. You know about the work I’ve done to help clean up PCs that were not purchased with Signature, and that this is a regular drumbeat for me here on Thurrott.com. (If not, please head over to the Microsoft Store, virtually or in person to find out more.) But the wider world is unprotected, and unaware of the benefits of Signature. And ultimately it’s up to Microsoft to do the right thing, by continuing to improve Signature based on feedback such as this, and by promoting Signature to its PC makers, retailers, and its customers.
Do I think Microsoft can and should go even further than that? Absolutely. But if the past five years have prove anything to me, Microsoft’s relationships with its customers are often complicated by its relationships with its partners. And in this case, PC makers have proven to be stubbornly unable to keep punching themselves in the face by shoveling crapware onto their PCs. The harm this does to the Windows ecosystem is almost incalculable, but like global warming it can no longer be denied with any credibility. Microsoft, please. Do not continue allowing your PC maker partners to ruin Windows.
I hope Microsoft does the right thing. But they already have the answer in Signature. They just need to push it harder than ever.