Microsoft Mounts Its Defense of Surface Reliability

Microsoft has come out publicly against Consumer Reports, stating that it disagrees with the publication’s Surface reliability findings. But it doesn’t offer much in the way of evidence.

“The Surface Team’s mission is and has always been to make devices that deliver great experiences to our customers and fans,” Microsoft corporate vice president—and the public face of Surface—Panos Panay writes. “It’s the motivation for everything we do, and we are proud of the Surface devices we have built.”

Windows Intelligence In Your Inbox

Sign up for our new free newsletter to get three time-saving tips each Friday — and get free copies of Paul Thurrott's Windows 11 and Windows 10 Field Guides (normally $9.99) as a special welcome gift!

"*" indicates required fields

This field is for validation purposes and should be left unchanged.

Yesterday, of course, Consumer Reports—a respected and well-established consumer products testing organization—removed its “recommended” label for all Surface products, citing their industry-worst failure rates. They based this decision on a survey of over 90,000 readers who own PCs and tablets using a methodology that has been tried and tested for automobiles and other products over many decades.

So this is a tough challenge for Microsoft. This isn’t a guileless tech blog or some ranting anti-Microsoft neophyte. It’s Consumer Reports. And what Microsoft offers in return to the organization’s real-world results is a bit of emotion and a hint at facts without any hard numbers.

So let’s examine every bit of evidence that Microsoft provides.

First, we get this interesting comment, which is a response to the Consumer Reports contention that an estimated 25 percent of those with Microsoft Surface devices will experience “problems by the end of the second year of ownership.”

“In the Surface team we track quality constantly, using metrics that include failure and return rates—both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book—are significantly lower than 25 percent,” Mr. Panay writes.

How much lower? We don’t know.

Does changing the time frame from “by the end of the second year of ownership” to “1-2 year failure rate” skew the results because more failures happen later in a product’s lifetime?

Also, he introduces the notion of “return rates” here. By definition, the feedback that Consumer Reports receives is from product owners, not those who have returned products. If someone is almost two years into device ownership, they are not returning the product. They’re just using it. And dealing with it.

So consider the issue muddled, in just one carefully-constructed sentence. Which I believe was crafted to confuse the issue.

But there is more.

“Additionally, we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1 percent,” Panay offers.

It’s not possible to understand how an “incident” relates to a “failure.” Mostly because he doesn’t explain the term. Likely because doing so would betray that this is an apples to oranges comparison.

“Surface also ranks highly in customer satisfaction,” he writes. “98 percent of Surface Pro 4 users and Surface Book users say they are satisfied with their device.”

I will point the reader to Welcome to Surfacegate, my description of Microsoft’s feeble attempts to ignore and then slowly fix endemic issues with those exact two Surface models. And anecdotally, I’ll point to the fact that the three Surface Book models I’ve used have all had reliability problems. And that, contrary to that, my Surface Pro 4 has never had any issues at all. Because that’s why the Surface Book and Pro 4 reliability issues are so vexing: Some never have issues, but others never stop having problems.

But the biggest issue I have with “customer satisfaction” is that it’s kind of a bullshit measurement when it comes to premium products.

As I’ve often said of Apple customers, when you buy something very expensive and it fails you in some way, you tend to cover it up because admitting to this failure betrays some mistake in your own decision making. For example, Apple customers bring their iPhones, Macs, and other products to the local Apple Store to get them fixed and then they brag about the company’s stellar customer service. Do Surface customers do the same? When one brings a Surface device in for service, and Microsoft asks them if they like the device, I bet a lot of people say they love it. Of course they do. They bought it. They’re probably still paying off the credit card bill.

On a related (and, yes, anecdotal) note, I suffer from this issue myself. I brought two Surface devices with me to Barcelona, because I still prefer Surface for some reason. But I just had a non-responsive keyboard issue, yet again, with Surface Book that required me to undock and then re-dock the Clipboard screen. This is the type of thing that Surface owners just get used to. My Surface Book experiences have been negative in so many ways. But I still really like Surface Book. It’s goofy.

“Our Surface Laptop and new Surface Pro continue to get rave reviews,” Panay adds.

Um.

Reviewers are patently unable to gauge device reliability, short- or long-term. So this has nothing to do with the conversation. In fact, Consumer Reports recommended Surface until they got the reliability data back. So you can see how this really works.

Ultimately, what we’re left with here is Microsoft doing what it must do, which is defending Surface. But this is also Microsoft offering no hard evidence that Consumer Reports was wrong to drop its recommendation of Surface.

So the conclusion here is obvious.

 

Share post

Please check our Community Guidelines before commenting

Conversation 67 comments

  • emanon2121

    11 August, 2017 - 6:49 am

    <p>My issues with the consumer reports data… </p><p><br></p><p>1) Data has been taken from 2014 and they do not break down failures by model. All is taken as a whole. From RT, to Pro, to Book. </p><p><br></p><p>2) Their definition of failure is quite broad. If there was a random restart does that count as failure? </p><p><br></p><p>3) Also, judging reliability on future or new products from old models can be a fools game. Surface Laptop is no longer recommended because of a tablet that Microsoft made 3 or more years ago. </p><p><br></p><p>Even with those points, I will say my wife's SP3 did have touch screen issues. We went to a store and they promptly replaced it. She said she would never buy another brand pc. There should be a metric that determines the time it takes manufacturers to fix current issues either at a software/hardware level or at a consumer level (replacing hardware). </p><p><br></p><p>Reporting issues is only halve the story. </p>

    • Gregory and Heather Steeno

      11 August, 2017 - 7:19 am

      <blockquote><a href="#165974"><em>In reply to emanon2121:</em></a></blockquote><p><br></p><p>1) Agree that it would be nice to have a more granular breakdown, by model and year, across all manufacturers, although some have a ton of model variations, vs just a few distinct ones.</p><p><br></p><p>2) Whatever their definition is of failure, as long as it was applied consistently across all OEMs from the &gt; 90K pieces of data, does it matter?</p><p><br></p><p>3) Until proven otherwise, the best predictor of future behavior is past behavior. I definitely look at historical reliability rates when purchasing a new(ish) car.</p><p><br></p>

      • emanon2121

        11 August, 2017 - 7:30 am

        <blockquote><a href="#165980"><em>In reply to gsteeno:</em></a></blockquote><blockquote><br></blockquote><blockquote>2) It helps to know what their definition is as to properly diagnose issues and to see if Microsoft's metrics are correct. </blockquote><blockquote><br></blockquote><blockquote>3) I don't expect you to buy a Ford F150 based on the reliability history of a Ford Fiesta and Focus. </blockquote><p><br></p>

        • Darmok N Jalad

          11 August, 2017 - 7:43 am

          <blockquote><a href="#165981"><em>In reply to emanon2121:</em></a></blockquote><p>Actually, a lot of people use brand reputation when buying a car. It speaks toward how the entire company is run. </p>

    • PeteB

      11 August, 2017 - 2:52 pm

      <blockquote><a href="#165974"><em>In reply to emanon2121:</em></a></blockquote><p>"Blah blah blah my wife's surface experience" is statistically meaningless. Nice try though.</p>

      • emanon2121

        25 August, 2017 - 2:31 pm

        <blockquote><a href="#166081"><em>In reply to PeteB:</em></a></blockquote><p>So is basing the reliability of an unrelated product from the unknown/unverified statistical history of another. But you know… logic. </p>

  • madthinus

    Premium Member
    11 August, 2017 - 6:59 am

    <p>The chickens has come home to roost.</p>

  • Nexus1216

    Premium Member
    11 August, 2017 - 7:07 am

    <p>Hey Paul</p><p><br></p><p>I mostly agree, but CR didn't exactly offer "hard evidence" either.</p><p><br></p>

  • Darmok N Jalad

    11 August, 2017 - 7:54 am

    <p>Yeah, the corporate response does seem rather soft on rebuttal. Sadly, it's fairly common in articles from AP, too, where the author bounces between using statistical data and statistical-sounding generalities to help prop up their argument. People sound more like trained politicians everyday. </p><p>Perhaps a stronger response from MS would have been to stand by their products by extending the warranty of their devices by another year. Apple did this on Series 0 watches due to battery swelling issues. It doesn't fix the problem, but at least the customer gets more support. It might also help MS nail down quality issues by getting a larger sample size of returns later in the product life. </p>

    • GT Tecolotecreek

      11 August, 2017 - 11:12 am

      <p><em>Perhaps a stronger response from MS would have been to stand by their products by extending the warranty of their devices by another year. </em></p><p>This action is the way to go for customers who purchased a premium product. Extend the basic warranty at least another year and longer for specific known issues that may appear intermittently. Apple, for some issues, has extended repair coverage for multiple years past the original warranty. If you pay top dollar the company shouldn't leave you holding the bag especially if they know specific issues may develop. </p>

    • Roger Ramjet

      11 August, 2017 - 12:36 pm

      <blockquote><em><a href="#165984">In reply to Darmok N Jalad:</a></em></blockquote><p>Agree. It's a statement move, show, don't tell. Extend the warranties, it's obvious, the right response on many levels, been done many times by others in similar situations, etc. This is a learning moment for Microsoft, and I have to think they will take the opportunity to show they are a serious bunch.</p>

  • JerryH

    Premium Member
    11 August, 2017 - 7:59 am

    <p>You are correct Paul; this is completely apples to oranges. When I first saw this defense posted yesterday I had to laugh. Comparing return rate and incidents to what CR terms "problems" is ludicrous. Many of the things CR is counting are never going to make it onto Microsoft's numbers because the person just deals with it. For example what you hit with your Surface Book keyboard not being responsive and having to disconnect the tablet and reconnect. CR would call that a "problem" – but you didn't report it to Microsoft and odds are very high that no telemetry captured it as an issue. So no way to compare Panos' numbers with CR's "problems" in any way. The defense was laughable and most anyone reading it will easily come to the same conclusion.</p>

  • rameshthanikodi

    11 August, 2017 - 8:20 am

    <p>"So the conclusion here is obvious" No.</p><p>Actually, return rates are a totally normal metric that <a href="https://www.cnet.com/news/5-reasons-microsoft-surface-shoppers-shouldnt-panic/&quot; target="_blank">other sites like CNET use</a>, and apparently Microsoft does too. So I don't think they "crafted to confuse".</p><p>I also believe Microsoft's IPU metric is roughly analogous to CR's "breakage rate", both are equally as ambiguous with their definitions undisclosed. Nonetheless Microsoft's IPU of 1% is a quite a large delta from CR's 25%.</p><p>Finally, the problems with the Surface Pro 4 were widely known and have been fixed since, but this CR's data is from that period. I really doubt the Surface Pro (5) has the same issues, so I don't think this un-recommendation should carry over for the new Surface products they introduced this year. They recommend the Galaxy S8 after the Note 7 afterall.</p><p>Oh and I can NOT believe Acer, of all brands, actually comes in third place. What the?</p>

  • barrywohl

    Premium Member
    11 August, 2017 - 9:02 am

    <p>I bought a Surface Pro 3 for my business several years ago. Docked it does fine. When I use it at home in the evening, I hate the Surface Pro keyboard. I upgraded to the Surface Pro 4 keyboard when it came out and got a little improvement, but not enough to keep me in the Surface family. The tablet function never meant much to me. I just replaced the Surface Pro 3 with a Lenovo X1 Yoga 2nd generation. No regrets. I will say that when I had booting problems with my Surface Pro 3, Microsoft customer service was just as good as Lenovo, and that's saying a lot.</p>

    • lvthunder

      Premium Member
      11 August, 2017 - 11:45 am

      <blockquote><a href="#165991"><em>In reply to barrywohl:</em></a></blockquote><p>If you weren't interested in the tablet function why did you buy a tablet in the first place?</p>

      • cheetahdriver

        Premium Member
        11 August, 2017 - 12:25 pm

        <blockquote><a href="#166024"><em>In reply to lvthunder:</em></a></blockquote><blockquote><em>Probably because like me, he wanted a tablet/notebook/desktop, a unit that could be everything in every situation. My experience mirrors his, and the replacement he picked is also the replacement I picked for the company. </em></blockquote><p><br></p>

    • Narg

      11 August, 2017 - 12:55 pm

      <blockquote><a href="#165991"><em>In reply to barrywohl:</em></a></blockquote><p>Only thing I dislike about the keyboard is the riser in the back. It makes the keyboard bounce as you type, and it takes away from good room that could be used for the trackpad and wrist rest. I wish they'd produce an older style version without the tilt riser.</p>

  • navarac

    11 August, 2017 - 9:04 am

    <p>Listening to Tom Warren's remarks on this subject on First Ring Daily, coming myself from the UK, I was interested to hear him say that {generic] consumer testing is taken less seriously her (in the UK). As he said, we have Which? magazine etc. but I must say that I have always taken any such reports with a pinch of salt. I would much rather rely on specialist reports, as certainly a lot of feedback will be from people that go off the deep end at the slightest problem. I suggest most people just quietly go about sorting any glitch out.</p><p>Of course, products need to be and should be fit for their intended purpose, although all have a built in obsolescence to some degree, even Windows 10 which will fail progressively on older devices.</p>

  • Juraj Kováč

    11 August, 2017 - 9:35 am

    <p>And that's the thing, really.</p><p>I love the Surface lineup, have a Surface Pro 2 as my primary PC and don't really see a reason to get rid of it just yet. Luckily, it's worked great for me since spring 2014. However, of the two people to whom I personally recommended the Surface Pro 3, both have had to send it in for replacement just months after purchase. It would just go dead all of a sudden. </p><p>I don't need to stress that Surface service and returns aren't really straightforward in Europe. Ultimately both units were replaced, but both encountered issues with the Type Cover later on. Microsoft refuses to handle these, and so your only option is to dish out another €130 for a keyboard, just two years in.</p><p>That's not a great track record.</p>

  • Larry Davidson

    11 August, 2017 - 9:54 am

    <p>I AM SHOCKED by the breakage rate reported by CR–not because it is so high, but because it is so low. I would have expected the failure rate graph to reflect a hockey stick. </p><p><br></p><p>I purchased my SP3 two years ago and in the interim, I have received five replacements from MS for defects covered under warranty (note: my MS warranty was extended beyond typical 1 yr. due to product failure within three months of receiving a <span style="color: rgb(0, 0, 0);">"reconditioned" replacement). That fifth replacement is now out of warranty, and defective, but still operational for the time being. </span></p><p><br></p><p><span style="color: rgb(0, 0, 0);">Just so you know, I baby my computers and none of the six problems were caused by me. Furthermore, no two defects have been the same and MS verified each machine as being defective prior to issuing me a replacement. Just saying.</span></p>

  • Skolvikings

    11 August, 2017 - 9:59 am

    <p>Anecdotally, I've had a new laptop every couple years over my 20+ year career. I've never had one with more issues than my Surface Pro 3. I had issues with the keyboard not working, issues where the network would sometimes stop working, necessitating a reboot, and my favorite, an issue with the touch screen where it would <span style="color: rgb(0, 0, 0);">randomly </span>act as though someone was repeatingly tapping the screen at one specific spot near the top-right corner. That last one would require a full power off and restart to resolve.</p><p><br></p><p>I won't pretend that I never had any issues with previous laptops, but none that were as prevalent or consistent as the ones I had with my Surface Pro 3. I also never bothered contacting Microsoft, because the issues didn't really start happening consistently until the warranty had expired, so why would I bother?</p><p><br></p><p>Count me as a person who had serious reliability issues with the Surface, but who doesn't show up in Microsoft's metrics.</p><p><br></p><p>PS. Apple customers can sometimes be an epitome of the sunk cost fallacy.</p>

  • rfog

    11 August, 2017 - 10:25 am

    <p>I gave up with my 3rd Surface Pro 3 and his keyboard, and need 2 Surface Pro 4 to get a real working one. I returned two surface dial because they died (one of them only worked for 4 hours).</p><p><br></p><p>I had to change 3 times my Microsoft Sculpt Ergonomic Keyboard.</p><p><br></p><p>Only that.</p>

    • Nonmoi

      11 August, 2017 - 1:34 pm

      <blockquote><a href="#166007"><em>In reply to rfog:</em></a></blockquote><p>You should look at the Wacom remote thing as a deploy ready alternative for the Dial. It's not as slick looking as the Dial, but way more particle.</p>

  • Bats

    11 August, 2017 - 10:42 am

    <p>Boy…all that stories that Panos said about him and his dad now seem to be B***S***. Like Paul said, Pants didn't really give a good defense of his product.</p>

  • nbplopes

    11 August, 2017 - 11:01 am

    <p><span style="color: rgb(0, 0, 0);">"But the biggest issue I have with “customer satisfaction” is that it’s kind of a bullshit measurement when it comes to premium products."</span></p><p><br></p><p>Why is it bushit?</p><p><br></p><p><span style="color: rgb(0, 0, 0);">1) "</span><strong style="color: rgb(0, 0, 0);">As I’ve often said of Apple customers, when you buy something very expensive and it fails you in some way, you tend to cover it up because admitting to this failure betrays some mistake in your own decision making. </strong></p><p><br></p><p>Well its bullshit because Apple users must have some problems regarding self confidence right?</p><p><br></p><p>I think its more likely to get fury from a customer that spent a lot of money on a product and had frequent problems with it than one that payed $300 for it and its expecting the problems out of the price. I think this is a far more rational way to see how human behave rather than has you think people behave. </p><p><br></p><p><span style="color: rgb(0, 0, 0);">2) </span><strong style="color: rgb(0, 0, 0);">For example, Apple customers bring their iPhones, Macs, and other products to the local Apple Store to get them fixed and then they brag about the company’s stellar customer service. </strong></p><p><br></p><p>Yes, customer service. Are you implying that the majority of Apple customers don't know the difference between good customer service and quality devices. <span style="color: rgb(0, 0, 0);">Only Windows users know the difference?</span></p><p><br></p><p>This looks like an attempt to turn something positive into something negative.</p><p><br></p><p><span style="color: rgb(0, 0, 0);">3) </span><strong style="color: rgb(0, 0, 0);">Do Surface customers do the same? When one brings a Surface device in for service, and Microsoft asks them if they like the device, I bet a lot of people say they love it. Of course they do. They bought it. They’re probably still paying off the credit card bill."</strong></p><p><br></p><p>So? It looks just the same right? You seam not to believe it. How does this than leads to conclude that CR metrics is BS?</p><p><br></p><p>Unless one secretly believes Windows users when they have a problem and is fixed, still report the problem while Apple users don't right? That is why CR reports are bullshit correct?</p><p><br></p><p>Yes, there is a difference between statistics of customer satisfaction and reliability issues. One influences the other but the first can be mitigated with good support … but only to certain extent. Because even if the device is fixed there is an associated cost especially if it is the go to work device, that is never forgotten, people just want the damn thing to work, not gifts or whatever. </p><p><br></p><p>Unless Windows users are a superior race of users comparing with Apple users this assessment does not make much sense.</p><p><br></p><p><span style="color: rgb(0, 0, 0);">4) </span><strong style="color: rgb(0, 0, 0);">On a related (and, yes, anecdotal) note, I suffer from this issue myself. I brought two Surface devices with me to Barcelona, because I still prefer Surface for some reason. But I just had a non-responsive keyboard issue, yet again, with Surface Book that required me to undock and then re-dock the Clipboard screen. This is the type of thing that Surface owners just get used to. My Surface Book experiences have been negative in so many ways. But I still really like Surface Book. It’s goofy.</strong></p><p><br></p><p>Right. So you are human afterall. Somehow CR did not get you in the repor. Got the most unemotional users from Windows and the most emotional from Apple hence the result, right?</p><p><br></p><p>Apple products have problems too. Ask any long term Apple user and he/she will confirm that. But that is not the point of the CR report. That is beside the point when comparing systems. Why ant Apple users seam to be fixed in this ideas is more a feat of MS tech marketing, culture even, than anything else. </p><p><br></p><p>PS: I don't live in the US. But in Europe, say Madrid when I go to an Apple Store its not full at all. In Portugal, In store Apple Spaces are usually empty if not for ocasional visitors looking and trying out devices.</p><p><br></p><p>I would expect that a city such as NYC with 8.5 million habitants, out numbering some countries, including European countries, adding the fact where Apple has say 60% or more of that market, store in high rotation places would be of course very busy. </p><p><br></p><p>Try to go to an Apple Store in Madrid, in Plaza Del Sol or in Passeig de Gràcia in Barcelona for instance, how busy is it in comparison (around 2.5 million habitants and I bet apple does not have 60% of that market)?</p><p><br></p><p>Look I'm not defending Apple, just looking at what you are writing and other and trying to be more rational about it. I believe in simple explanations, usually the first one are the simplest regardless if they fit our liking or not. Usually trying to find other explanations because we fell bad about the initial ones, somehow, usually is a rabbit hole with no end because it comes from a forcing a result mindset rather than looking at the moment.</p><p><br></p><p>Cheers.</p><p><br></p>

    • Roger Ramjet

      11 August, 2017 - 12:20 pm

      <blockquote><em><a href="#166014">In reply to nbplopes:</a></em></blockquote><p>Lordy. Take a chill pill man, you completely misunderstand what the writer is saying and went off on a tangent; running with bull.</p><p><br></p><p>The write up has very little to do with Apple, he isn't attacking Apple or its customers, or using it to minimize the Microsoft issues. It's a passing reference that he used to illustrate the very common "investment" psychological hypothesis, because Apple is apt here since Apple products are in the category Surface aspires to be, he then went further to apply to himself also using his Microsoft product saying he behaves the same way. He is using this to counter the Microsoft point in their response. </p><p>His main point in this write up he is entirely attacking Microsoft, point by point, and somewhat uncritically backing up CR </p><p><br></p>

      • nbplopes

        11 August, 2017 - 2:03 pm

        <blockquote><a href="#166032"><em>In reply to Roger Ramjet:</em></a></blockquote><p><br></p><p>The author my have all the issues he feels with MS response. Never refered to it as bulshit but lets call it bushit for the mentioned reasons.</p><p><br></p><p>But also refers to the CR report as bushit for premium products. For that he uses … as you refer, psychological issues. But we dont know the kinds of questions were done to the 90.000 consumers. But i find it hard to believe that people would avoid simple and direct questions such as "have ever used warranty service" "if you have how many times" … so on and so forth.</p><p><br></p><p>So with no data whatsoever the writer is implying that the gap might as well be minimum between Apple values and MS values.</p><p><br></p><p>Why ?</p><p><br></p><p>So bulshit + bulshit we get is bushit. </p><p><br></p><p>To that I call it mudying the waters. Different styles but stil …</p>

      • nbplopes

        11 August, 2017 - 5:48 pm

        <blockquote><a href="#166032" target="_blank"><em>In reply to Roger Ramjet:</em></a></blockquote><p><br></p><p>On another note, take a deep look at this chart: <a href="https://www.thurrott.com/wp-content/uploads/2017/08/chart.jpg&quot; target="_blank">chart.jpg</a></p><p><br></p><p>What can be said? Instead on going on speculating on the psychological phenomena to justify such gap …</p><p><br></p><p>People expect premium devices along with innovation to be more reliable than non premium devices. At one corner we have Apple, the right place considering that the company almost exclusively bets on premium devices quality blabla, its place its a natural one to be. It is expected.</p><p><br></p><p>It would be expected MS to be close to the same place considering that the company also exclusively bets on premium products and innovation … but is really really far. </p><p><br></p><p>This created a problem for the Windows PC industry. Paul should be happy because its no longer the one saying that MS should bet strong on reliability, both in with Windows 10 and Surface devices instead of trying to speculate that the values are not closer because of some psychological phenomena that leads to Apple users to be more forgiving than Windows users while he, himself is an example of such forgiveness in the Windows camp.</p><p><br></p><p><strong>The problem is this. Does this mean that Premium Windows PC's overall have a robustness problem leading to people no wanting to shell out more money to get those devices in the Windows ecosystem? Or is this a problem localized only with MS or does it span the entire Windows PC industry?</strong></p><p><br></p><p>Its quite obvious that other companies in the chart do not focus only on premium Windows PC devices. In fact, the largest volume of sales of these companies seam to be in medium to low end devices, not premium. </p><p><br></p><p>People are far more lenient to system glitches in cheaper devices than in premium devices. Thats is why, I think, companies like Toshiba, DELL, Lenovo, Asus or HP are not getting the same heat as MS seams to be from CR.</p><p><br></p><p>To ascertain the problem the chart should be segmented in low end, medium and high end. Mind you, the gap between Apple and MS would not be closer because of this segmentation considering both companies focus on premium products. But it would be interesting to know if the other companies come closer to Apple, closer to MS or remain in the same position. when premium devices are concerned. Again this would not affect either Apple to MS results most probably, but could clarify how fairly are Premium Windows PC devices compared to Mac Premium PCs.</p><p><br></p><p>That is the only thing that this graph does not give us an idea off when reliability is concerned. But that does not mean its bullshit of course when it comes to comparing at the moment user experience reliability on both Apple and Surface devices.</p><p><br></p><p>Cheers.</p><p><br></p><p><br></p>

        • Roger Ramjet

          12 August, 2017 - 12:07 am

          <blockquote><a href="#166113"><em>In reply to nbplopes:</em></a></blockquote><p>Again, you misunderstand the writers point. In two places above you say he was "justifying the gap, or "trying to speculate the values are not closer", but absolutely not, that was not his point. His point was actually nearer to the opposite, he was using this to contest Microsoft's response. He was saying maybe Microsoft users who reported high satisfaction to Microsoft were afflicted with this malady. Yes, one could take his example of Apple users being forgiving of some flaws as some sort of "justification" in a very thin skin reading, but then he immediately used *himself* as an example of such a person, based on his *Microsoft* product, so that should immediately dispel that, besides this "sunk cost" or "investment" psycology being very common in literature, it should be obvious in most evenhanded readings that this is just a reference to that idea, using an example that is at hand. </p><p>As to your other point, maybe yes who knows. We can speculate but I agree we don't have data. I think one of the very reasons Microsoft launched Surface is, the mainline premium segment, where a lot of the profit is, and which they belatedly found out creates the mindshare and goodwill (but be warned, not when your product breaks every other day), when new inflection points arise, had been seded to Apple by the Windows industry in favor of beige boxes as OEMs competed with each other to the lowest common denominator, so hard to say whether the chicken came before the egg on that.</p><p> I will say though yes, a premium product should be reliable, but reliability isn't the only factor, and based on Microsoft product strategy they weren't going for a 1 to 1 reliability vs. Apple(you can't make things that do more, have more moving parts, and less designed-in form factor protection and expect them to be just as reliable as the already well made alternate), its just that the approach seems to have bitten them on the behind with how badly that metric has turned out in this report, and based on real world complaints of users. In fact, as I have speculated elsewhere in these replies, the mechanical designs of the Surface Book &amp; Surface Pro alone predict these results vs other *Laptops* (camshells), aside from additional software issues that folks have experienced, which part of that is most certainly that Windows 10 tries to do a lot more than MacOS, right, you have touch and the legacy trying to live side by side. So, Microsoft should have known unless they were being unrealistic and foolish they would not be as reliable as&nbsp;incumbent Apple product, hence their strategy all along could not have been that they expected to be at the same place on that aspect. </p>

          • nbplopes

            12 August, 2017 - 4:13 am

            <blockquote><a href="#166134"><em>In reply to Roger Ramjet:</em></a></blockquote><p><br></p><p>Read it again … you may be correct.</p><p><br></p><p>My point of confusion was the "bulshit". Paul was referring to "customer report" referred by Panos not the CR report.</p><p><br></p><p>Two days working nokn stop on a.prpject … people make mistakes.</p><p><br></p><p>Sorry for the confusion.</p><p><br></p>

            • skane2600

              12 August, 2017 - 1:33 pm

              <blockquote><a href="#166140"><em>In reply to nbplopes:</em></a></blockquote><p>You admitted you might be wrong. That's a rare thing around here. Well done.</p>

              • nbplopes

                12 August, 2017 - 5:45 pm

                <blockquote><a href="#166228"><em>In reply to skane2600:</em></a></blockquote><p><br></p><p>I have no problems admitting it when I detect a mistake of mine about anything. Thank you Roger Ramjet for calling my attention to it.</p><p><br></p><p>Thank you too for the observation.</p>

    • bsd107

      Premium Member
      12 August, 2017 - 3:53 am

      <blockquote><a href="#166014"><em>In reply to nbplopes:</em></a></blockquote><p>"Apple products have problems too. Ask any long term Apple user and he/she will confirm that."</p><p><br></p><p>You know what, actually, Apple users do not do this.</p>

  • Breaker119

    Premium Member
    11 August, 2017 - 11:01 am

    <p>Aside from the actual reliability issues – Consumer Reports should <u>never</u> be trusted for electronics reviews. They have never been good at it. Reviewed products are too limited, etc….and not just for computers. </p><p><br></p><p>Last time I really used them (before I cancelled my subscription) is when I was looking for a mid-size SUV. In that case they made their recommendation, but only had like 6 or 7 vehicles reviewed. I get that they have to buy all of the products they review, but if you're going to make a recommendation based upon limited information you should defer the recommendation</p>

    • cheetahdriver

      Premium Member
      11 August, 2017 - 12:22 pm

      <blockquote><a href="#166015"><em>In reply to Breaker119:</em></a></blockquote><p>The reliability reviews are based on survey results from members. I take these surveys every spring and fall, and my Surface experience should have been rolled into these results for years. This isn't just based on their own usage.</p>

  • John Jackson

    11 August, 2017 - 11:53 am

    <p>Sounds like we should be asking for 3 year warranties.</p><p><br></p><p>What amazed me was that the best rate was Apple's high of 10% crap! </p><p><br></p><p>Expensive crap at that MSFT and AAPL.</p>

  • lvthunder

    Premium Member
    11 August, 2017 - 11:54 am

    <p>To be fair Consumer Reports didn't define their metrics either. I mean what do they define as a "problem".</p>

    • Narg

      11 August, 2017 - 12:56 pm

      <blockquote><a href="#166026"><em>In reply to lvthunder:</em></a></blockquote><p>Don't you have to buy their magazine to get the metrics?</p>

  • cheetahdriver

    Premium Member
    11 August, 2017 - 12:21 pm

    <p>CR isn't terribly good on electronics, mostly because they are an "after the fact" measurement, ie, the surveys are from people who have already owned the unit for a year or two. That does make them the gold standard for customer satisfaction (as opposed to JD (we can make a survey so narrow that your product will come out on top) Powers), but knowing what the customer satisfaction rate in a SP3 may be of limited use to the buyer of the new SP5 (or SP). CR's results match my experience with multiple SP3s and a single SP4 (we never deployed them). My company is no longer entertaining any Surface product for deployment and we are depreciating the units we have, moving them to non-mission critical locations. If you feel like past performance for the entire Surface line is a prelude to future issues (to paraphrase the stock folks), then this is a big red flashing light. </p><p><br></p><p>Certainly it was enough for us to drop Surface. We have gone to Lenovo X1 Yogas (bought through the Microsoft Store for the Signature Experience). Our experience with the Stores have been exemplary, I can't say enough good things about them. Dropping Surface doesn't mean you have to drop the store experience. </p>

  • Narg

    11 August, 2017 - 12:52 pm

    <p>My Surface boots, and is fully usable every time. I'd call that reliable. Is it perfect? well no, it does suffer from the WiFi slow issue. But it is reliable in operation at least.</p>

  • Roger Ramjet

    11 August, 2017 - 1:01 pm

    <p>So here's the thing: imagine the conversation when Microsoft first went to their OEMs and said you know, we should be making "Toaster-Refrigerators", the objections would have been things like: 1, there is no market for this 2, it will be too expensive 3, you can't make this reliable, etc, but I am pretty certain these 3 would have been among the key objections. </p><p>So, this is a pottery barn thing. Since Microsoft decided to go ahead anyways, (and got to enjoy the accolades), they must own up to the downsides of the process if they hope to build this brand. Actions, that's what is needed here. And over time they will get it exactly where they want it – learnings and improvements are already obvious – the brand will prosper. </p>

  • bbold

    11 August, 2017 - 1:34 pm

    <p>Yes, and we learned with Consumer Reports recently that their "scores" and lab testing aren't up to snuff when they said literally the same thing about Apple. Apple got them to change their stance, I'm not sure how, maybe Microsoft could do the same? All I know is that at the very least, this is extremely BAD PR for Microsoft right now. They'd better find a way to fix it. Stock value may suffer, as well.</p><p>Another insight: Apple's stellar customer support (just swapping out laptops for new ones in Stores) has spoiled everyone.</p>

  • Wizzwith

    11 August, 2017 - 2:01 pm

    <p>And Consumer Reports has a satisfaction rate of 1/5 stars LOL https://www.consumeraffairs.com/online/consumer_reports.htm ;D &nbsp;</p>

  • Derek Belrose

    11 August, 2017 - 2:01 pm

    <p>I can only speak from my experience. I have tried 4 surface books and a surface pro 4 in the last 18 months. The first two surface books (i7) experienced many of the problems that are known today such as the hot bag issue, hardware detaching of the screen and the display driver constantly crashing.</p><p><br></p><p>The second two were performance base surface books. Both of them bulged at the battery in the screen in less than 24 hours. You could feel the metal conforming around the bulge. </p><p><br></p><p>The surface pro 4 was great but too small for my needs.</p><p><br></p><p>Microsoft's customer support at the store near me was phenomenal. Each time I went in, they recognized me, listened to my issues and tried to help. The issue wasn't customer support, it was the reliability of the machines (or just the size of the screen). </p><p><br></p><p>I love the idea of the Surface Book, I bought it 4 times…i still want to buy it. Batteries overheating and bulging are not issues that I am going to risk any longer. Not for $2900 machines.</p>

  • Waethorn

    11 August, 2017 - 2:35 pm

    <p>I have a problem with the "1-2 year return rate" part.</p><p><br></p><p>The thing is, if it's past the first year, it's out of warranty. Who is going to return something out of warranty for a repair? What do repairs cost from Microsoft for out-of-warranty units? Or do they even do them?</p><p><br></p><p>I'd bet that probably at least some of the people interviewed would have dead units outside of the one year warranty that would just say "screw it, I'll buy a new one cuz it's gonna cost too much to fix". Because customers of mine do the same thing – and for systems of all ages.</p><p><br></p>

    • Tallin

      11 August, 2017 - 3:05 pm

      <blockquote><a href="#166075"><em>In reply to Waethorn:</em></a></blockquote><p><br></p><p>I imagine most people purchasing a Surface get the Microsoft Complete coverage, which covers it for two years.</p>

      • skane2600

        12 August, 2017 - 1:28 pm

        <blockquote><a href="#166084"><em>In reply to Tallin:</em></a></blockquote><p>Extended service contracts are generally considered a bad deal. Then again, if a product has a tendency to fail more often than average …</p>

        • Waethorn

          14 August, 2017 - 12:30 pm

          <blockquote><a href="#166224"><em>In reply to skane2600:</em></a></blockquote><p>It depends who they come from. From an OEM, a 3 year warranty is usually only $99. Big box stores gave extended warranties a bad name by offering in-store warranties with far worse terms for more money. </p><p><br></p><p>It's insurance. And people pay for insurance plans on their phones all the time.</p>

      • Waethorn

        14 August, 2017 - 12:28 pm

        <blockquote><a href="#166084"><em>In reply to Tallin:</em></a></blockquote><p>Keep imagining that. Consumer Reports even said extended warranties on consumer electronics are not a good deal.</p>

  • Wizzwith

    11 August, 2017 - 2:52 pm

    <p><span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">According to CR's data here, the AVERAGE problem rate&nbsp;across ALL the brands in the survey is 20%.&nbsp;Yes, 20%!!&nbsp;If you aren't skeptical of CR's methods at that point than you just aren't giving it any thought.&nbsp;&nbsp;</span></p>

  • Bdsrev

    11 August, 2017 - 5:05 pm

    <p>Pauls words are always good for a laugh 🙂 the breitbart of tech blogs</p>

  • red.radar

    Premium Member
    11 August, 2017 - 5:42 pm

    <p>Consumer reports leads with their reputation and offers data. They are doing so transparently. Microsoft offers bluster and language. Come on Microsoft show your hand. Either you are holding aces and you can win or you are bluffing.</p><p><br></p><p>even if the data is overly generalized by CR. You can Pareto the failures and show us what is most likely nuanced software issues vs show stopping hardware defects. </p><p><br></p><p>Data show me the data. Talk is cheap everyone does it these days </p><p><br></p><p>you are are positioning yourself as a cloud services data analytics company. Lead and show us. We know hardware is t going to matter to your bottom line. </p>

  • hrlngrv

    Premium Member
    11 August, 2017 - 5:45 pm

    <p>Let's parse MSFT's statements a bit more closely.</p><ul><li><em>predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book</em></li></ul><p><br></p><p>What about other Surface devices? Actually, admitting issues with earlier devices and pointing to improvements in manufacturing would be positive, wouldn't they?</p><ul><li><em>we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1 percent</em></li></ul><p><br></p><p>This seems to support the hypothesis that earlier Surface devices were flakier than the current ones.</p><p>MSFT clarifying by admitting Surface Pro 3 and earlier were flaky but the latest models are much more reliable? It'll never happen. MSFT never willingly admits errors.</p>

    • nbplopes

      15 August, 2017 - 6:34 am

      <blockquote><a href="#166112"><em>In reply to hrlngrv:</em></a></blockquote><p><br></p><p>People only report hardware problems in case of total failure. For instance, would. MS accept a return if the keybpard did not work properly. Most people go to a forum, see others with the same problem and wait for a fix as MS is saying … next time, next time, now all is sorted … of course it is not. If the fix does not come time passes, return period passes, People might send it warranty, comes back … same problem …. than 6 or 8 montjh have passed the consumer is stuck ….</p><p><br></p><p>This all software as a service … tranferred to hardware … is a total con. Repeat … a total con IMHO. We dont know when hardware ends and software starts and vice versa. Consumer position is weakened to Corporations wishes and needs while paying for it, people dont understand what they signed up for as their lifes are not ruled by tech per tech, people are people.</p><p><br></p><p>Could we have a CAS, becausr this is not the way to go. I dont want this for a future. We need Companies need to be accountable for their products irrepective how weakned is someone heart in a negotiation.</p><p><br></p><p>My older child with an SP2 is constantly disconnecting and connecting the keyboard. I bought him a USB keyboard tk fix th issue … bad that is not the point is it?</p>

  • joel12

    11 August, 2017 - 6:11 pm

    <p>Have had two of micro soft pads ,biggest waste of money , never touch them </p>

  • Winner

    11 August, 2017 - 6:43 pm

    <p>…because marketing is always better than delivering good products…</p>

  • beatnixxx

    11 August, 2017 - 7:02 pm

    <p>My Surface Pro 4 has been extremely reliable. No problems, no issues, just works. It's only been about a year, so maybe I'm about to fall off a cliff statistically, but so far I've been very pleased.</p>

  • Cain69

    11 August, 2017 - 10:33 pm

    <p>I have bought Dell, Lenovo, Samsung, Razer and MS Surface Pro / Books.&nbsp;I have had PCs from ALL these OEMs work for YEARS without a problem – and yet others crap out in months.&nbsp;PCs are tools – and they will break – I EXPECT THEM TO!&nbsp;What I care about is the service when things go wrong!&nbsp;MS has the BEST service I have ever received!!&nbsp;</p><p><br></p><p>I have switched my business and my family over to MS Store PCs.&nbsp;And have not regretted it for a minute.&nbsp;Reliability does not matter to me if they stand behind their product with great service.&nbsp;AND NONE of the Wintel OEMs have come close to the level of MS’ service and support!!&nbsp;</p>

    • hrlngrv

      Premium Member
      12 August, 2017 - 5:53 pm

      <p><a href="#166132"><em>In reply to Cain69:</em></a></p><p>FWIW, I've never had poor service from either HP or Dell. Acer, OTOH, well . . . the less said the better.</p>

      • Cain69

        12 August, 2017 - 11:30 pm

        <blockquote><a href="#166273"><em>In reply to hrlngrv:</em></a></blockquote><p>I have bought about 50 Dells and about 30 Surface Pro/Books. I can write a book on my experiences with Dell's "Premium Service"! My experience I guess is different then yours. I am happy Dell / HP has worked out for you. </p>

  • wright_is

    Premium Member
    12 August, 2017 - 5:38 am

    <p>My SP3 failed after 18 months, it had one of the dodgy batteries. What I think is a very important part of the equation is how a manufacturer reacts when something fails. I've had some refuse to acknowledge problems or make you jump through hoops (I'm looking at Apple and its iPhone replacement policy – you have to send it away for weeks for inspection and repair and you don't get a loaner during that time, with my first iPhone, it went back 3 times for a total of 6 weeks in the first 7 weeks of ownership, before they agreed to replace it; over here there are no Apple stores and repairs are only done by authorized repairers by post, most Android suppliers provide a replacement device when they pick up the defective one).</p><p>In the case of my SP3, it failed a couple of weeks before the story about the faulty batteries broke. I emailed MS support and they didn't quibble, they just organized for a replacement to be sent and the old one to be picked up. For me that is as important as the overall reliability of the devices – any device can fail, no matter who the manufacturer is.</p><p>It doesn't give MS a free pass to make unrealiable kit, but that can happen to anyone – in the old days, we rolled out over 100 Compaq LTE notebooks to our sales staff, we had over 20% DOA and a 60% failiure rate in the first 3 months. It turned out there was a faulty batch and, while we had ordered a large number of devices at one time, we got a lot of machines from the faulty batch. There were various hardware problems and the case cracked under normal day-to-day stresses. Again, s*** happens and Compaq replaced all machines with new ones from a different batch, defective ones and ones that hadn't shown any problems, no jumping through hoops, they just shipped new notebooks and arranged for the old ones to be picked up.</p><p><br></p>

    • hrlngrv

      Premium Member
      12 August, 2017 - 5:48 pm

      <p><a href="#166142"><em>In reply to wright_is:</em></a></p><blockquote><em>. . . </em>over here . . .</blockquote><p>You might want to be explicit where you are. Not everyone who reads this comment of yours would have read your previous comments.</p>

      • wright_is

        Premium Member
        14 August, 2017 - 5:19 am

        <blockquote><a href="#166271"><em>In reply to hrlngrv:</em></a></blockquote><p>Germany</p>

    • hrlngrv

      Premium Member
      12 August, 2017 - 5:51 pm

      <blockquote><a href="#166142"><em>In reply to wright_is:</em></a></blockquote><p>FWIW, the first Zotac ZBox I bought had a dud harddisk. I bought it through Amazon, and Amazon didn't hesitate to replace it. Dunno how it would have been if I had bought it direct from Zotac.</p>

  • lordbaal1

    12 August, 2017 - 2:05 pm

    <p><span style="color: rgb(0, 0, 0);">Consumer Reports is BS.</span></p>

    • hrlngrv

      Premium Member
      12 August, 2017 - 5:56 pm

      <p><a href="#166243"><em>In reply to lordbaal1:</em></a></p><p>Most surveys are BS without the details of response rates by category.</p>

  • Angusmatheson

    14 August, 2017 - 3:14 pm

    <p>The question here is whether past reliability of surface devices which all report is poor should be used to judge the future reliability of surface devices. Especially the Surface Laptop which must had been virtually absent in the consumer reports survey. If I was consumers reports, I think it would have been more fair to not take the buy recommendation away from the Surface Laptop until they had more information on that device, given their in house testing was good and to remove the buy recommendation from the devices that they have data on their reliability – Surface Pro and Surface Book.</p><p><br></p>

Windows Intelligence In Your Inbox

Sign up for our new free newsletter to get three time-saving tips each Friday

"*" indicates required fields

This field is for validation purposes and should be left unchanged.

Thurrott © 2024 Thurrott LLC