Microsoft Has A Software Quality Problem

Posted on October 6, 2018 by Mehedi Hassan in Windows, Windows 10 with 114 Comments

Earlier this week, Microsoft released the Windows 10 October 2018 Update to the public. After months of testing with Insiders, the company finally started rolling it out to the general public, giving them a taste of all the new features. Except…there is a major issue: the update is deleting some users’ personal files, with no apparent way of recovering the missing files.

More than 48 hours after users started reporting the issue, Microsoft pulled the October 2018 Update while it investigates the issue. And that highlights a fundamental problem with Microsoft’s testing process for Windows.

You see, Microsoft mainly relies on the Windows Insider program for fixing bugs on the Windows software. As Microsoft develops new updates like the October 2018 Update, it rolls out new features over weeks and months, which is then tested by Insiders. Most of the widespread problems with these new features and updates are usually noticed by Windows Insiders and later fixed by Microsoft before a major update is released to the public.

And then there are issues that only affect some users, leaving them buried under all the other feedback and bug reports within Feedback Hub, the app used by Windows 10 users and Insiders to report bugs and suggestions to Redmond. That’s exactly what happened with the data deletion issue. Some Windows Insiders have reported this issue as early as 3 months ago, as noted by Rafael Rivera on Twitter. If you simply open up Feedback Hub and search for “documents deleted” you will see multiple reports from Insiders where their files were automatically deleted after they installed the update.

Because these issues aren’t widespread, most of these reports from Insiders had low levels of upvotes, ranging from 3-10. That means Microsoft likely never even noticed these reports from Insiders, which is why is the only thing that can explain how such a major issue made it to the public release — all of that simply because of how the Insider program’s bug reporting system actually works.

Important issues like these are often buried under thousands of other general feedback like “Make File Explorer look more modern” or “Add Acrylic to File Explorer”. Although these are valid feedback, they often prevent actual issues from being noticed by Microsoft, and that is a fundamental problem with the entire Insider program system. Things get even worse considering the fact that Microsoft has cut down on independent testing a few years ago, relying mostly on Insiders for testing its software.

Microsoft’s Windows Insider team will probably find a way to explain itself here, but it’s more than apparent: the Feedback Hub simply isn’t capable of dealing with issues from millions of users, especially when the app is mostly used by fans to provide feedback — mainly fueled by their love for the OS. Critical issues are not only buried under borderline-fictional things like users asking for Acrylic on File Explorer, but they are also often buried so deep down that Microsoft’s systems never even pick them up, leading to disasters like this.

Tagged with , , ,

Join the discussion!


Don't have a login but want to join the conversation? Become a Thurrott Premium or Basic User to participate

Comments (114)

114 responses to “Microsoft Has A Software Quality Problem”

  1. wolters

    They are not fully utilizing the feedback hub. It is amazing that they have such a successful insider program and these things still happen.

  2. ianhead

    Even surely just having the ability to report on the severity of an issue must be beneficial for helping to triage these issues. It's worked for the open source community for a long time.

    The Insider program is also, belying its name, too damned open. I can't imagine how many bug reports that are just crap like "Bring back Aero glass!!!" and bug reports of malicious behaviour that would be correctly attributed to user stupidity. Because you can start alpha testing Windows releases with the click of a button. It's insane.

    • evox81

      In reply to ianhead:

      This was my thought as well. Give people a way to mark the severity of an issue.

      Of course, "severity" is subjective and everyone thinks their problem is the most important one, so it would likely devolve in to exactly the same thing we have today.

    • jimchamplin

      In reply to ianhead:

      There are dozens of not hundreds of threads exactly like that. The Aero Glass ones get absurd numbers of upvotes. The neophyte mistakes get treated seriously by the other newbies.

      There was one I remember: X button doesn’t X out of <some application> when question is up!!!!! Help!!!!!!

  3. bbold

    With the variety of devices, operating systems, drivers and such in the wild today in regards to PC usage, one can almost understand Apple's "walled garden" approach to technology. Since Microsoft does NOT isolate the user from the outside world (since most of us aren't using Windows 10 in S Mode), Microsoft and we users will continued to be plagued with these issues from within Windows 10 Home and Pro, and they always have, dating back to 3.1 or earlier. This is a problem that will never go away, and in a way, it's a vote for the 'walled garden' approach of controlling every aspect of a user's experience. It's either full-on S mode or nothing at all, I guess. While we do not want to live within a walled garden as PC users, you can see the problem by living in that world. Microsoft can't detect every known issue, and based on the small amount of upticks or votes on this in Feedback Hub, I'm guessing this file deletion issue is affecting a small percentage of people, and just happened to get a bit of media attention. Am I wrong? Do we have a number of people affected?

    • skane2600

      In reply to bbold:

      An OS update can delete user files just as easily when it's Windows S being installed as it is if it's full Windows being installed. Obviously an OS update isn't subject to any S-mode restrictions.

      If Apple has any advantage it's that they make 100% of Macs with fairly limited options so they don't have the diversity of platforms that Windows must support. The only way that Microsoft could enjoy the same limited platforms would be for them to make all Windows PCs themselves. That ship has already sailed.

      It really has nothing to do with "walled gardens".

      • hrlngrv

        In reply to skane2600:

        . . . If Apple has any advantage it's that they make 100% of Macs with fairly limited options so they don't have the diversity of platforms that Windows must support. . . .

        True, but is that particularly relevant to OS upgrades deleting user files in user profiles/home directories?

        I figure this Windows upgrade glitch will turn out to be due to an obscure group policy setting. That's something MSFT has given Windows which few other OSes have, and it may turn out it's not a strength for Windows.

        • skane2600

          In reply to hrlngrv:

          I never claimed that diversity of platforms was the cause of this particular problem, I merely pointed out that it was a challenge Apple didn't have to face.

          I never realized that there was a "delete user files" group policy setting. That would be obscure.

          • hrlngrv

            In reply to skane2600:

            It's not called delete user files. it's called Computer Configuration/Administrative Templates/System/User Profiles/Delete user profiles older than a specified number of days on system restart. It defaults to not configured, so if it's one cause, users would have had to have changed the default setting, so users would have contributed to their own problems. OTOH, unless this setting just appeared in the past few months or no one changed its default setting before this, this hasn't been a problem in the past, so why now?

            • skane2600

              In reply to hrlngrv:

              So you speculate that it's the users' fault because a symptom for this problem that hasn't been described by Microsoft (i.e. user profile deletion vs missing some files) could be caused by setting a group policy that can't be set for Windows Home and rarely is set outside of an enterprise and hasn't been identified as part of the root cause is, in fact, the cause.

              • skane2600

                In reply to hrlngrv:

                You were just giving some credence to 3rd party speculators. Given the problem has occurred on Windows Home which doesn't support group policies, this can't be the root of the problem. I think you're just trying to spin this into evidence against Windows' overall design rather than a software bug that could happen due to sloppy update work on any OS.

                • skane2600

                  In reply to hrlngrv:

                  It's highly unlikely that many people using Windows Home and "hacking" their way into installing the Group Policy Editor. As far as registry entries are concerned, the setting HKLMSoftwarePoliciesMicrosoftWindowsSystem for Windows Home 8.1 is simply default, no value set. I can't speak to Windows 10, but there's no reason for it to be any different.

    • irfaanwahid

      In reply to bbold:

      I'll agree with you if this problem has not affected any Surface devices at all.

  4. EZAB

    Does anyone know if this is also a problem on Windows 10 Insider Preview Build 18252 (19H1)? Fast and Skip Ahead.

  5. hrlngrv

    Perhaps MSFT is using government-like cost-benefit analysis, and the benefits of preventing a few deaths (or profile cleansings) just isn't worth the costs of tracking down and fixing the causes. The only way I can see to fix that is to make certain egregious software fubars are subject to law suits, ideally really bloody expensive class action ones. From my perspective, MSFT would be substantially improved if it had to face a lawsuit brought by thousands of users, and even more improved if it wound up needing to pay out in 10-figures. IOW, the cost of fubars would be more clearly fixed in senior management and outside directors' minds.

  6. navarac

    A similar issue found (although not so far reaching in seriousness) was 19H1 build 18252 borked the Office 365 installation. It (O365) had to be re-installed or full-repaired. My report in the hub on that got a response from JenG so at least I'm confident that it is being looked into.

    • derylmccarty

      In reply to navarac: My O365 borked on startup and first run of 252. After three tries to open Outlook and with a response "to try again" I was willing to go to the Feedback Hub and wail. But I also shut down then restarted as normal and all has gone well without any intervention. So I just put it down to a glitch. I should have reported the issue then, but will "add" to the "problem" on feedback.

  7. FalseAgent

    the thing is, this just shows that users do actually identify and report the bugs. The role of 'insiders' is still sound. The problem here really lies within Microsoft.

    • paulc543

      In reply to FalseAgent:

      I'm sorry, but I'm not following your logic on several fronts. First, non-beta testing end users should not be the ones identifying and reporting bugs. In this case, real people lost real, valuable data. A beta tester, AKA "insider", signs up and knowingly accepts the possibility of data loss in exchange for the ability to test, use and influence upcoming features. Your average non-"insider" Windows user has made no such agreement, and this situation is more dire when Microsoft takes it upon itself to install an upgrade or update without any user consent.

      As far as the soundness of the roll of the "insiders", this is a glaring example of the entire system having failed miserably. Either not enough "insiders" set up their systems in such a way as to be effected by, and report, this bug, or their reports were drowned out by, as Paul touched on, far more people using the program and reporting system to request non-critical stuff. I've tried to leave feedback numerous times over the years about critical issues I've experienced where either data or time loss has occurred. It's always drowned out by fanboys talking about dark themes and other non-critical, mostly aesthetic BS.

      A real beta testing program should be limited to select and mostly professional users. Opening it up to anyone and everyone guarantees that real issues will be lost among the noise of users testing it because they think it's cool to be testing it.

      This whole fiasco reflects poorly on both the "insiders" and Microsoft for both the way they've set up and run the program and that they've relied on such non-paid, volunteers for the testing of a critical product.

    • hrlngrv

      In reply to FalseAgent:

      Insiders may report bugs, but they don't provide exhaustive system configuration information. Windows update ate my files is a report, but without knowing all nondefault settings, how would MSFT engineers be able to diagnose much less fix the problem?

  8. jimchamplin

    I quit trying a long time ago. Why bother to report an issue that nobody will care about? I mean, it’s just something like HDMI audio out not working, or unusually high CPU usage, or THE ENTIRE UWP ENVIRONMENT GETTING INTO A STATE WHERE NOTHING RUNS! NOT EVEN SETTINGS!!

    You know?

    The devil’s in the details.

  9. ZeroPageX

    There's a downside to the way most do agile development. Bugs get buried by features as you said. This is happening everywhere, and everyone talks about how great agile is (there are fantastic benefits), but this needs to be addressed. Over time, if not kept in check, quality goes out the window. Bugs don't get fixed until they reach critical mass.

    The Start Menu search index has been broken since the initial release. This is basic functionality, but it's not deleting user data or making the PC physically explode, so they prioritize new shiny bits nobody will use.

  10. Steve Martin

    The process needs to reflect more than "how many people reported the problem." This bug is in the category of "SHOWSTOPPER!" No release until this is fixed. If there is no way to prioritize based on the harmful effects of a bug, they've missed a fundamental rule of QA.

    • hrlngrv

      In reply to smartin:

      This is where professional QA staff is a lot better than unpaid part-time testers. Sometimes the severity of a bug takes precedence over the frequency of the bug manifesting. Most Insiders aren't qualified to assess severity objectively.

  11. ChristopherCollins

    I truly wish they would just do monthly security updates (unless something warranted an out of band fix). One release a year is fine.

    The Feedback Hub is a freaking disaster. Too many voices and it's so focused on the enthusiast market that it misses much of what we need on the IT/Business side. I'm an enthusiast too and I've reported issues that never even see an upvote. Many of the insiders just want glitz and glamour added to it. I'd prefer they strip it down to a Windows 10 Core for Business model.

    They don't get an accurate install sample from people or businesses with older hardware in my opinion.

  12. rfog

    In reply to rfog:

    As I had full security copy (and my real critical files are in a second m2 disk as a matter of safety), I did the install and... It went too fast and the result is my version is still 1803. Seems they "voided" the installation or simply it failed.

    Many thanks, guys, for your advice.

    I have to say that I updated two more PC without any trouble before being aware of this problem, but, as this one, my documents are not inside standard documents folder.

  13. Tony Barrett

    ...and so, another disastrous PR event for MS and consumer confidence in Windows falls to an all time low. Another botched update where MS ignored reams of reports by their 'insiders' (aka, unpaid b*tches), and released another poorly tested update onto the waiting masses. MS should now take stock, and before this hideously broken bi-annual testing/update service actually starts trashing more machines than it successfully updates. From 2019, they should revert to a single, annual update, probably released in the fall, allowing them twice as long to tune the release and fix these terrible bugs before people start throwing in the towel on Windows.

  14. blcktrgrp

    good content helpful thank u festival

  15. mazharnoor

    Nice Article, Thank for Sharing.Make Money Online Fast

  16. epguy40

    hi Mehedi. here's a recent article that you may want to check out from ZDNet:

    seems like Microsoft needs to get back to the "fundamentals"

  17. mocymam

    I have lost my My Documents files and I didn’t have a backup... MS support has told me they are no longer helping after they tried many resolutions that have failed. So now.. can someone point me to an attorney that has fought MS in the past, or show me a class action lawsuit involving this issue?


    This is why I left Windows just before 8 came out. I've been on macOS since Mountain Lion, and every annual update has been trouble free except for minor issues. (And I have always updated on day 1 or 2) Mojave has been the best of the lot, indicating that Apple is improving their testing processes. This troubles me because I did like using Windows from 3.1 for workgroups through 7, and I would like having the option of switching back if I have to/want to. I don't believe in being a "fan" of software or hardware -- computers and their OSs are just tools. But the way things are looking I'm staying with my Mac -- not that I feel trapped... I suppose I could be mocked for this, but my all time favorite Windows OS was 3.11 for workgroups with the Norton Desktop -- I feel nostalgic just thinking about it. (95 was incredibly hyped but was buggy as hell - I can't count the number of times I wiped the hard drive & reinstalled) I just loved booting my computer and loading various reference and content CDs in the 3.11 days. Who needed the internet?

  19. waethorn

    "Software-as-a-Service"? Heh.

    Next time I agree to this with Microsoft (which is probably never), my terms are that Microsoft provide an SLA.

  20. waethorn

    People ask me why I criticize Microsoft's code quality as of late. And why I don't use Microsoft products anymore....


  21. waethorn

    Someone out there can probably find some reason to defend Microsoft's action to lay off their internal QA testing team in favour of a public beta test. There is somebody, right?? I mean, besides the bean counters looking at Microsoft's labour costs - or that nutjob Satay Nutella.

  22. stlbud

    Who didn't see this coming? Nadela comes in to Microsoft and fires all the quality testers. Users have been conscripted into being beta testers. Why is anyone surprised at this kind of catastrophe? You're right, Paul, the Insider Program and Feedback Hup are not working.

  23. RM

    They need to allow categorization of reports based on the type of report (new feature, enhancement, break/fix, security issue, etc.) and severity (New Feature, Minor Improvement, Broken Feature, Lost Data, Security Risk, etc..) Then they can concentrate people on reviewing feedback properly. As long as they have enough knowledgeable people to help with this.

    • hrlngrv

      In reply to RM:

      Poor MSFT employs no one capable of devising a text filter for problem descriptions that looks for the tokens delete or empty or gone?

  24. mdkimzey

    Microsoft does have and has had a major problem with updates.   It is always a good idea to ensure that you have backups prior to any update.  Since many updates are forced (installed whether you want them or not), daily backups are a good idea.

    It is not unreasonable for users to expect that their files will remain following an update. 

    Windows 10 have shown a steady history of poor quality.  When computers are a vital tool in your business, you can't easily afford excessive downtime.  When 1803 came out, workstations that used NVIDIA GPU boards went from stable to regular BSODs every 5 to 10 minutes. The solution did not involve Microsoft Support, Forums or NVIDIA Support.  The solution came from a highly skilled tech. at our company and a third party tool (not from Microsoft) to analyze system dumps.  We were down for one week.

    Although the workstations were working and stable prior to the 1803 update, Microsoft considered this a problem caused by a product from a third-party. 

    The 1809 update shows that Microsoft turns a deaf ear to their clients.  This sounds harsh but the messages from Microsoft that I am seeing are:

    • We know what's best for our users.  We will force updates on them for their own good. 
    • Get the updates out!  It's patch Tuesday - any problems?  We'll fix those later.
    • Microsoft uses minimal testing.  Interaction between Microsoft and the top used third-party driver are not considered as important.
    • Microsoft leans heavily on the "Users accepted the EULA" and accompanying risks argument. The users signed up for it...  Microsoft  takes little-to-no risk for releasing bad updates.
    • User workstations are fair game - like this really awesome sandbox. 
    • Don't worry about the users - it takes a lot of effort to switch from Windows.

    The absence of respect that Microsoft has for their clients is very clear.  There is a big difference between releasing an update and letting it escape.  No change will be forthcoming from Microsoft.  It will take a sustained drop in stock price before they really take action.  The need a Quality Control program - one that actually works.  I don't see an overhaul coming any time soon.  An overhaul is deeply needed.  But, they will blow this off (as they did the 1803 issues) and, at-most, try temporary fixes.  An overall process change (that takes client feedback into account) is badly needed.

    • Winner

      In reply to mdkimzey:

      I'll add to this. Microsoft's DNA is to look out for themselves and not care a whole lot about the users. Especially consumers. Which is probably a big reason why Apple has taken so much of the consumer space and Microsoft has failed so miserably there.

      I don't say that as a joke. Actions that point to this DNA:

      • They dumped Plays for Sure where customers lost purchased music that had DRM
      • Some customers lost "purchased" exercise videos on XBox when it didn't fit Microsoft's directions
      • Microsoft offered "free", then pushed, then coerced, then FORCED many users through deception to upgrade from Windows 7/8 to this new polished gleaming "free" lipstick-on-a-pig Windows 10 OS that added ads, popouts, and forced upgrades. They cared about THEIR OS growth targets and not about what users were asking for (or not asking for)
      • Windows 10 and new versions of Office made "save" and "save as" less convenient by putting OneDrive (another company push) front and center in the save options so that users would more likely use Microsoft's cloud services for their monetization strategies
      • When Alexa and Siri and Google Assistant were becoming things, suddenly Cortana went from free to included to non-removable as part of Windows 10. This wasn't because users wanted it, it was because it served Microsoft's paranoia about THEIR need to not be left out in the cold with another new wave in computing

      Microsoft uses its users as a body of people to try and preserve its corporate stature and maintain revenue targets. It doesn't create delightful products that consumers clamor for (like Apple, mostly), it creates me-too products and services and then tries to leverage its other products to cram these new things down our throats.

  25. brduffy

    Its hard to imagine a more serious bug than "My files are being deleted against my will". Is there no way to classify a submitted bug as critical? I mean, shouldn't there be a critical bug list that MS pays particular attention to?

    • irfaanwahid

      In reply to brduffy:

      I still feel getting rid of or reducing the number of testers at Microsoft was the worst idea. They should still test the critical/core functions of the OS.

      Deleting files is a big issue and I fail to understand how Microsoft could not see this themselves, let alone blaming on Insiders.

      For me, Insiders should have always been kept as an added layer for extra testing.

      The kind of attention they give to Azure and testing.. if a small fraction comes to Windows.. things will improve.

  26. tehhellhound

    I just delayed my features updates by 365 days and quality updates by 7 days(you don't want to fall too far behind on security updates, but this gives us enough time for the suckers to test it first). I think I am going to call this the Windows Outsider program. Honestly, the only reason I use the operating system is for a gaming machine and all I wanted to do is be able to run my games and production software for my stream. I couldn't care less about the stupid features, especially if they might break my system and make me have to work in fixing it on top of my job and have my broadcast off the air until I do.

  27. sentinel6671

    I hope (although at the same time doubt) that the Windows team is doing some serious soul searching here. Two feature updates a year, when the capacity to do testing is lacking to say the least, is unsustainable. The level of idiocy and Kool-Aid consumption Microsoft demonstrates in pushing forward on this Windows as a Service path just boggles my mind. Biannual updating, without proper, exhaustive testing, doesn't serve users and it definitely doesn't serve the platform. If the features added were worth something, that might make the pain tolerable. But, for the most part, they're not!

    I've been a Windows user for nearly 25 years and I've never been thinking more than I am now about how to leave the platform.

    As for the Windows Insider Program, I'm sure there are good, well intentioned people participating, but sadly it's a reflection of the wider internet, too much noise and not enough substance.

    • hrlngrv

      In reply to sentinel6671:

      The very cynical part of me wonders whether MSFT fired its QA department just before it committed to upgrading Windows twice a year because a serious QA department would only have gotten in the way of the brave new rapid release regime.

  28. Reldel

    My Ninja Cat is functioning very well on 1809.

  29. Stooks

    Microsoft has a testing/reporting process problem.

    Stooopid users that go out of their way to install a big update they day it comes out without a proper backup have way bigger problems.

  30. rseiler

    "Microsoft’s Windows Insider team will probably find a way to explain itself here."

    Yes, but feet should be held to the fire this time if it's the usual fluff. For all the reasons mentioned in the post and the comments, something needs to give--and before 1903.

  31. matsan

    Since most open-source projects have devoted, unpaid members doing triaging of issues reported one would expect Microsoft to have the same. They could at least have one intern sifting through incoming feedback! Sadly this doesn't seem to be the case.

  32. rfog

    In reply to rfog:

    My ThinkPad is just ready to reboot. It will reboot this night if I don't reboot now. Great, MS, great!!!!!

    • SvenJ

      In reply to rfog: You do have the option to backup files before that happens. Surely you did that already though.

    • tsay

      In reply to rfog:

      Go to SettingsUpdates and SecurityWindows UpdateAdvanced Options and on that screen you'll see an option Pause Updates.

      Hit that and Larry's not only happy, he's ROFLMAO...and you'll be too. ?

    • DocPaul

      In reply to rfog:

      Something that might work is you could plug in a USB drive with a bootable Linux distro like Ubuntu and reboot straight to that. From within Ubuntu you'd have access to all your local storage and could back it up before you allow windows to reboot and finish updating. And not sure which Thinkpad you have but maybe you could pop the battery out to buy yourself time to prepare for this.

  33. Lauren Glenn

    Reddit would probably be a better view of bugs. Why go to the trouble of having your own hub when Reddit does the job just as well, if not better?

    But in terms of this not being widespread... it's probably because people like me put all their files in cloud backup solutions where the documents aren't stored in the Documents folder. I didn't really notice until I saw Brad Sams's post on it with Twitter.

  34. bart

    The Feedback hub needs to become a two way street. MS needs to prioritise the serious problems reported by Insiders and brought to everyone's attention when publishing a new build.

    Stuff like acrylic, though important, needs to be taken up by the team, but no need to be brought to users attention. Maybe a quick 'we're working on it' in the Feedback Hub will suffice IMHO

    • AnOldAmigaUser

      In reply to Bart:

      One would think the vaunted AI would be able to discern that "my documents are gone" is really serious and should be investigated.

      Absolutely agree that the Feedback hub needs to go both ways. Again, if someone reports that their files are gone, someone from Microsoft should actually contact them and try to troubleshoot the issue. I am sure that there is nothing in the upgrade log stating "deleted user files", but just the idea that someone could lose their documents during an upgrade should have alarm bells ringing, and someone contacting the affected person within one business day. Causing data-loss is the cardinal sin of IT.

    • navarac

      In reply to Bart:

      You are right. But "Stuff like Acrylic" is not important and is just eye-candy as for instance on the log-in screen of Redstone 5. I could get the same effect by taking my spectacles off ! As Paul has said more often than not, there needs to be a concentration on the consistency and de-bugging of the OS rather than nonsense such as Acrylic etc.

  35. navarac

    One problem may be that (as Insiders) we have installed one build upon another build without an issue, or a clean install and tested that. What has been missed is probably testing Build 17763 of 1809 (Redstone 5) being installed over the last Redstone 4 build (1803). (BTW there were 34 builds)

    • Dan1986ist

      In reply to navarac:

      Hit the nail on the head there. Release Preview should have been used to test the 17134 to 17763 upgrade before 1809 went out to everyone who checked Windows Update prior to it being pulled, but it wasn't.

  36. RobCannon

    I think separating feature requests from actual bugs would help. As an insider, a bug should have more fields for reproduction steps, impact, etc. The insider team should track those bugs as part of their release gate.

    • SvenJ

      In reply to RobCannon: It does to a degree. You can capture reproduction steps, include screenshots, etc. There are decent options to provide information for bugs. Part of the problem is that Insiders aren't 'testers', as is mentioned, they are typically fans. I have seen MS's collection process/engine and it is technically impressive. I am not sure how the 'AI' extracts truly significant issues with low counts out of chaff with super support. It certainly isn't just volume that triggers reaction. Insiders can assign feedback as suggestions or bugs, but there are a lot of cosmetic inputs labelled as bugs. Providing Insiders a way to assess the significance of 'issues' would likely result in as much subjectivity as simply bug/suggestion. Certainly this issue would have deserved the highest level of significance, but I guarantee there are insiders that would characterize Notepad not having a dark mode as just as severe.
      Bottom line is that MS needs a robust QC dept/division re-instated. Still, there is no way they could test on every possible configuration of hardware and software Windows runs on. This could conceivably slipped by internal testing as well. We will likely never know, but it would be interesting to find out what is common among those that suffered data loss. (I had done just one system with no issues myself. Will likely do a recently acquired Surface Pro (2017) [can we call it 5 now?], since it doesn't have much important on it yet)

  37. Vitor Canova Weingaertner

    I believe another problem with the feedback app is that people use to post duplicate bugs as new instead of upvote. I think in this case you have 100 feedbacks saying files were deleted but all with 1 upvote. O pray for the tool they use to classify be more clever than this.

    • skane2600

      In reply to Vitor_Canova_Weingaertner:

      While I imagine the system can be improved, creating such a system that really works is difficult. How many times have we seen on StackOverflow that it suggests existing questions that are supposed to potentially answer the question you are forming, only to find that those existing questions are irrelevant, focusing on a few words in your question.

      Although I'm skeptical of today's "AI" hype, perhaps this is an area where MS could apply its AI technologies.

    • navarac

      In reply to Vitor_Canova_Weingaertner:

      The consolidation of feedback in Collections was supposed to address that. Obviously it isn't.

    • mryves707

      In reply to Vitor_Canova_Weingaertner:

      Totally agree. Feedback hub is a mess. They should work hard on it to make it a valuable and central tool for their testing process. Especially when relying on insider feedback. They should improve categories and search and include a field with severity of the feedback.

      Feedback hub should be also the central tool for every feedback to Microsoft products and services. E.g. try to give feedback for Dynamics CRM.

  38. jblank46

    I think while Paul believed that 1803 and 1809 were a type of "quality update", they really weren't. I think Microsoft ran out of features to add in so it appeared as if they were applying a more measured approach. Microsoft truly needs quality updates for all of its software right now. Office 365 desktop apps suffer similar quality issues, particularly Outlook.

    • Demileto

      In reply to jblank46:

      Or, more likely, they're dedicating the majority of their human resources to finish Windows Core OS instead of Windows 10 as it exists today.

      • jblank46

        In reply to Demileto:

        Agreed. I think Windows and Office are suffering for different reasons. Windows is suffering because Core OS is the priority now while Office is suffering because the pace of updates are furious for these legacy applications. They're trying to graft the agility of the web onto these legacy applications and it shows if you are on the Office 365 Current branch. This is where Google Apps shines because they do not have the legacy baggage to contend with.

        Then there's Edge, which I have really tried to make my main browser but it is so buggy and the user experience is so poor compared to Chrome that I gave up. I've also had the clipboard break on 1803, where I cannot copy and paste content between UWP and legacy apps that only a reboot solves. How they managed to break something as basic and fundamental as the clipboard is puzzling, but the only good theory I can come up with is that Windows is on life support while they work on the next thing.

    • bart

      In reply to jblank46:

      Your interpretation of quality is different from what Paul meant. That said. Valid points

  39. unfalln

    Every feedback system I've used has a rating for the "impact" of a bug. E.g. whether it has the potential to lose data on the user's behalf. The only reason one would not include a rating such as this is if the reporter is expected to enter incorrect data.

    • SvenJ

      In reply to unfalln: And that is likely very accurate. I can tell you that there is a lot of disparity when just looking at bugs and feature requests in the feedback hub. Giving Insiders the ability to rate Notepad not having a dark mode on a sliding scale of severity would likely not provide a lot of valuable data.

  40. arknu

    Well, it would help if the Feedback Hub actually worked. For some inexplicable reason, Microsoft separates feedback by country. So when I search, I only get results for feedback from my country, not the rest of the world. So instead of upvoting existing feedback, I get to file my own, because I can't find existing feedback. Thus, you get an enormous amount of duplicates.

    • dvdwnd

      The regional silos are a disaster. There are still no collections in my region (well, maybe two in the wrong language), so reporting anything is completely futile. I have given up. The Insider program mostly looks like a marketing ploy and the Feedback Hub is a digital void where frustrated users can shout to feel better.

      But ninjacats.

  41. mebby

    In reply to Demileto: I used surface pro as my everyday tablet and just bought the Go to replace an SP4.
    I do experience the problem on the Go with browsers being unresponsive to touch. I also have had the problem with SP4 using RS4. Sometimes I have to restart app and a few times the SP4 or Go. However, this has been very rare. Luckily none of my PCs went through the update to RS5 before they pulled it.

  42. Koshy

    I think Microsoft considers "Home" users as beta testers for Enterprise versions on Windows and Microsoft could argue that this time the "Home" version of windows successfully stopped the bug from being rolled out to enterprise and hence there is no reason to change their QA methodology.

    • Tony Barrett

      In reply to Koshy:

      MS considers everyone using Win10 as their own personal beta testers. They collect so much information, MS can't call it any other way.

    • skane2600

      In reply to Koshy:

      Enterprises are relatively locked-in to Windows for all the well-known reasons, but Enterprises also aren't so foolish as to judge this as anything but a serious QA failure and Microsoft's leadership is naive if they think this won't tarnish Windows reputation with Enterprises.

  43. cr08

    I think what is heavily alluded to here, and is often implemented on any other issue/bug tracker system, is having a clear delineation between actual bug reports and features requests is what is sorely needed with the Feedback Hub. Currently it is just WAY too simplistic and ironically in that state makes it hard to use at times and causes situations like this. In addition to making that delineation, they need a simple moderation staff that can find similar/same bug reports and combine them into one single report. Since they are relying so heavily on votes in the current system, having the same thing reported numerous times with votes spread out between them is just no bueno.

  44. Stokkolm

    They should separate the feature requests from the bug reports in Feedback Hub. That would probably help a lot.

    • gregsedwards

      In reply to Stokkolm:

      They already do this; although it could arguably benefit from being made a little clearer. You can launch Feedback Hub manually and create a new entry. The very first option reads "Report a Problem" or "Suggest a Feature." The option you choose automatically designates your feedback accordingly. However, most modern apps have a "Submit Feedback" option somewhere in the UI, which launches Feedback Hub and creates a new entry tied to that app. It's the identical feedback form, but here you're first asked to describe the feedback, and then in step 2, you have the option of tagging your entry as a problem or suggestion. When viewing feedback in the Feedback Hub, there are big pivots to show all feedback vs. my feedback, and then within those views, there's a filter control, which you can easily set to show only problems or suggestions.

      Either way, it's more of a problem with selecting testers and educating them about their responsibilities around reporting issues and writing feedback. The Insiders program being completely open and opt-in probably doesn't help the quality of feedback the Microsoft is going to receive. I know a lot of "Insiders," who see it as simply a way to get a "perpetually free" copy of Windows 10 to use on their computers. Other than the telemetry they tacitly allow Microsoft to track, they're probably never going to take the time to submit actual bugs, because that's boring and requires commitment. Instead, they're going to get used to buggy builds, ask for cosmetic features, and then gripe about how crappy Windows 10 has become.

      You can't outsource quality control.

  45. chriswong13

    Quality has been an issue for Windows 10 since the beginning. Plenty of quirky/buggy issues in many areas come and go with each release of Windows and core apps, like Mail, etc.

    • Darekmeridian

      In reply to chriswong13:

      Software quality has always been an issue for Microsoft not just in Windows 10 (Patch Tuesday isn't a new concept it's been around long before 10) . But on the other hand I think the work they do covering so many different configurations and drivers is a pretty amazing feet of engineering anyone who has worked on a large programming project knows how hairy things can get. There is no reason that anyone should be loosing files in an in-place upgrade, that's kind of the whole point and it's clear that it can be done because they have done it in the past so there are not a lot of excuses for this kind of bug.

      With all that said, there is a certain amount of "user responsibility" at play here. Anyone who legally drives a car needs insurance, not because we are expecting to have an accident but just in case we do have one. I don't know of anyone who's not had a hard-drive go out at an in-opportune time or had Word crash in the middle of that college report that you have been working on for 5 hours and discover auto-save wasn't turned on.

      I think Microsoft should work on some better tools for disaster recovery and backup that are buit into the system. OneDrive is good but it's a first step to something that should be more comprehensive.

  46. Aaron44126

    I'm sort of glad that Windows is progressing more quickly than it used to, but I think that they're going a bit too fast... I'd rather see yearly upgrades (like Apple), maybe each one could have a little more in them than these six-month upgrades, but they could also get more time to bake before release.

    Anyway. I'll just sit on the Semi-annual NOT targeted branch and get these three or four months after everyone else has had time to spot and figure out the issues...

  47. AnOldAmigaUser

    Does the screen still say "Your files are just where you left them."? Personally, I found it worrisome that this message ever appeared.

    Maybe they should force the upgrade to every corporate computer, and the personal devices of all employees, starting in the C-Suite, before releasing it on the world. They should use the tools that they use to roll it out to customer, in that the corporate users should have to seek it out and install it without IT support.

    My guess is that the quality of the upgrade process would improve.

  48. siv

    Yet again we get a bug that shows MS do not have any actual testers on the payroll. There have been issues with pretty much every Windows 10 feature update. I was hopeful that now Windows was being handled by what Paul refers to as the "Adults" in the Server and Azure teams, we might see some quality improvements and straight away the first release handled by them actually has a much worse bug than the previous mob. What happened to Microsoft? I am sure under Gates or Ballmer if this had happened once, it would have been stamped on immediately and heads would have rolled!

    I can't believe that Nadella is sitting back and paying no attention to this stuff, I know he is all about Azure and services, but if your main platform for delivering these services is being damaged so badly by these sorts of issues I would be implementing real change in the way testing is done. I think the insider program is a great way to get feedback, but in no way should it be used as a free replacement for real testers.

    After all Microsoft is a "software" company, and software 101 is make sure it works to a very good degree and absoloutely will not lose user's data. Come on Microsoft get back to tried and tested principals, this is embarrassing!


  49. BBoileau

    I left the Insider program out of this very frustrating element. On my everyday computer, which I don't use the Insider preview, both the 1803 and 1709 broke the machine and I had to install fresh via iso file. I intended to install this update in this manner and I am thankful I had some personal things that delayed this. As Microsoft moved away from the fluffy crap of the earlier WaaS updates, it seems they didn't adjust the focus of the Insider Feedback Hub. It might not be a bad idea to consider.

  50. roastedwookie

    Nadellasoft quality...I am curios what Brandon has to say about this. Their insider program is nothing but a way for MS to avoid paying some dedicated testers, giving zero to none about quality! Pathetic

    • EZAB

      In reply to roastedwookie:

      I'd like to know what Gabe Aul has to say about this? Maybe he should be back in charge of the Insiders Program like he was. Something needs to change with the program!

  51. Thomas Parkison

    Well no shit, Sherlock! I could've told you that.

  52. John Craig

    This is the second messy rollout in a year. I wouldn't be surprised to see heads rolling in the next few weeks. Dona Sarkar is probably, and rightly, quite nervous about her immediate future in the insider programme.

    I guess Microsoft will just be thankful that massive enterprise clients won't be anywhere near rolling out 1809, and the percentage of affected personal users/consumers is probably small, but still, it's gotta hurt to see the insider programme fail twice in a year.

    • waethorn

      In reply to John_Craig:

      Oooo, but pretty icons and fashion and unicorns....

    • Koshy

      I don't think Satya an Azure guy and in turn Microsoft cares about Windows and moreover the consumer "Home" Versions of Windows, to see heads roll. As far as Microsoft is concerned "Home" users are beta testers for Enterprise versions of Windows and "Home" has successfully does it's job this time.

  53. A_lurker

    The real problem is the lack of a proper internal testing team. An internal team can develop test plans based on the specs used to write the code and then try to break the code, they have access to the internal documents. Also, there can be some generic tests that are run on all releases to check for install and driver problems. External testing has major flaws. No access to documents means no idea of what to test or how to test it. Lack of training, most external testers are not trained testers which is a specialized skill set that not everyone has nor or will they be good at it (I am developer who works closely with professional testers). There is no easy way for external testers to directly communicate issues with correct programming team so bugs are not always assigned to anyone. Here an internal tester is more likely to know which team is responsible for the code and can, in theory, communicate directly with them. I know the testers I work with communicate directly with me on all problems they find so I can fix them. Plus external testers are likely to much more skilled and technically adept than the average user. So having backups or data on separate partitions is more likely than with them than with the average user for example. Thus some problems will likely not be seen by the external testers as often as they will be seen by the average user.

    The internal testers will not catch everything but with a solid team and good protocols the issues that do get released to the wild should be minor annoyances not crippling disasters for users.

    • waethorn

      In reply to A_lurker:

      Too bad Satay Nutella let them all go before Windows 10 shipped in favour of the public beta test, nay, sh*tshow called the Insider Program.

    • skane2600

      In reply to A_lurker:

      I agree. No way an outside group can as effectively test a release as well as professional testing group can. This also illustrates the weakness of relying on unit testing alone internally.

      • hrlngrv

        In reply to skane2600:

        Thing is to really concentrate the minds of MSFT senior management and outside directors, it's necessary to produce serious financial consequences. Deleting everything in a user's profile without prompting for permission to do so seems to me an ideal example of something which should FINALLY place limitations on waivers of merchantability and fitness found in all software licenses. Deleting user files without seeking permission to do so should trigger strict liability. Having to pay out 10-figure damages wouldn't overly damage MSFT, but it just might teach MSFT's powers that be that cheap and short-sighted cost cutting can lead to very expensive mistakes.

  54. eshy

    Microsoft might want to put the internal Windows Insider team's focus back on finding issues with Windows, in most blog posts from that team it seems their trying to build local communities and do other things that aren't related to testing

  55. paulc543

    The problem is one of Microsoft operating on the assumption that they know precisely and exactly how users are using their systems. Be it rebooting to install updates regardless of what the system might be doing at the moment, assuming that if the mouse/keyboard isn't being operated, the computer is idle with nothing opened or being processed, or assuming no one stores any files outside the designated folder structure, Microsoft acts as if a user's system is Microsoft's to manage. It is not, and I fear it will take a class-action law suit to get Microsoft to wake up to this fact. If Microsoft is going to disallow a user to meaningfully control their system, then Microsoft should be fully liable when they cause the loss of work/files/time to the end user.

    Because the applications I need to run are only available on Windows, I'm unfortunately a captive user. But issues like this are severely degrading my opinion of Microsoft and will result in my leaving the platform at the first available opportunity.

    • pachi

      In reply to paulc543:

      While I don’t disagree with the conclusion. I think the reason is more that Microsoft has barely any QA nowadays and that’s why seemingly every twice yearly update has major major issues. We play this song and dance every big release. Ridiculous.

  56. pachi

    I’ve long felt that the feedback hub really just satisfies some internal bureaucracy at MIcrosoft.

    Obvious bugs get lots lots of upvotes but on the other hand they’re obvious and MS doesn’t need the hub to address them.

    Actual bugs and problems often get just a handful of upvotes, and are very difficult to even search and find in the Hub and rarely if ever are addressed by Microsoft.

    But... it’s a really easy way for the, admittedly very kind and nice, Microsoft employees to wave away criticism or bug reports. I see it on Twitter all the time “did you file a report in the feedback hub?”

  57. rina mandal

    Should have better software testing program to improve quality.