Windows 10 – where’s the testing? Or do they care?


(This post was meant to be a comment to the “This Is Why Windows 10 Is So Unreliable” article but I got errors posting, so I’ll put it here).

I, and apparently quite a few others looking at the Feedback Hub and Windows 10 support forum, found after 1903 came out that the Calendar app refused to see any iCloud calendar events. I reported it in early June and it was finally fixed (fingers crossed) around the time of the last Patch Tuesday, although what exactly fixed it is guesswork – there’s no mention of the issue in any release notes as far as I can tell, but there is a mention in the Insider Preview 18963 in August:

“We fixed an issue resulting in iCloud calendars not syncing with the Calendar app.”

hidden away at the bottom, looking at first glance like it hardly effects anyone and “who cares?”

Surely even with limited telemetry there are enough people using (or trying to use, when it works) the Calendar app along with iCloud? You’d think Microsoft would test this combination properly before release and then not have to take 4+ months to fix the bug for us plebs who don’t need Outlook. I can’t decide if it’s down to broken testing or a lack of interest in their own built-in apps.

Comments (21)

21 responses to “Windows 10 – where’s the testing? Or do they care?”

  1. navarac

    Reading "This Is Why Windows 10 Is So Unreliable", it's all down to automation and the inexorable reduction of human resources!

  2. epguy40

    well nkhughes check out this article from ZDNet titled "Why is Windows 10 a Mess?"

  3. codymesh

    there's no telling where the problem actually lies, it could very well not even be a Windows issue, but a backend one, where windows telemetry doesn't necessarily help.

  4. Lordbaal

    They fixed it. So what's the problem?

    It takes time to look at the millions of line of code to see what broke it.

  5. wright_is

    “We fixed an issue resulting in iCloud calendars not syncing with the Calendar app.”

    hidden away at the bottom, looking at first glance like it hardly effects anyone and “who cares?”

    Surely even with limited telemetry there are enough people using (or trying to use, when it works) the Calendar app along with iCloud?

    On the other hand, iPhone users make up, what, 16% of the smartphone market. And a lot of iCloud users will be Mac users and those that do use Windows, how many use the Calendar app and how many just use the web browser? At a guess, I'd think that iCloud calendar in combination with Microsoft Calendar app will affect a single digit percent of overall Windows 10 users.

    The other thing is that services like iCloud are a moving target. If Vendor A doesn't like Vendor B, they can easily tweak things to make sure it doesn't work with Vendor B's products, especially when they are cloud based. There probably has to be a dialogue between Apple and Microsoft about the protocol - maybe iCloud was using a new, undocumented feature and Microsoft either had to analyse it and re-engineer it or wait for feedback from Apple.

    Then there is priorities. A calendar not working is a very low priority, compared to the latest zero day security issue, serious bugs in frameworks etc. that affect a large part of the user base.

    I don't know where the bug count currently is on Windows 10, but I remember going through MSDN when Windows 95 was launched and there were, literally, thousands of open bugs with the platform. It took years for many of them to be addressed, because they were low priority - not serious, not affecting a lot of users.

    Heck, I reported a bug in Visual J++ in 1998 and it was still there last time I looked.

  6. justme

    <sarcasm> Well, I am sure Microsoft has plenty of telemetry that suggests people arent using the Calendar app to sync with iCloud </sarcasm>

    If I had to guess, it is down to the Calendar team is small, automation, and given the complexity of W10 - a large list of other important fixes that are deemed a higher priority. When you consider the enromous amount of legacy code Windows carries, even small changes have the potential to cause unintended (and perhaps, unconsidered) consequences upstream. I think it would be an interesting study to sit down and figure out how Microsoft "racks and stacks" problems it addresses as well as looking at how many fixes caused issues elsewhere.

  7. Winner

    Microsoft is more focused on steering you to their additional cost services such as Office 365 and OneDrive with plenty of storage. Their QA group was disbanded many years ago and instead the insider program is supposed to be their quality control.

  8. jimchamplin

    Well, I could bring up the single pixel border at the top of UWP apps being incorrectly shaded...

    But I've been harping on that for over a year and it's still there in 1909. No, they don't give a good goddamn.

    At least it's not macOS Catalina.

    • wright_is

      In reply to jimchamplin:

      Not having the resources for a low priority "visual" bug is very different to not caring. If they are all busy fixing security issues, an inconsequential cosmetic issue isn't a high priority. Also, I've never noticed it (and just launched a couple of UWP apps to check), maybe it is a UWP / video driver issue?

      I'm not trying to defend Microsoft here, just point out the realities of development and prioritisation of bugs.

      • jimchamplin

        In reply to wright_is:

        I understand that reality, which is why I more joke about my harping on it than anything.

        Its spotty on my systems, even. I see it on Intel graphics (Mac mini late 2012) and ATI Radeon mobile (Thinkpad X140e) but not on my Thinkstation S20 with a GeForce 1030.

        Its weird. I have a very strong feeling if I had a machine newer than 7 years old I wouldn’t be seeing it.

        My point about it being better than Catalina still stands though ?

  9. hrlngrv

    Tangent: a different problem which I've seen first hand and a few have reported: give the taskbar full transparency, and that fubars clicking on your user picture (icon?) on the left side of the Start menu. The submenu doesn't appear, meaning its text is invisible, but it's possible to click on the invisible items. This has been around for at least the last 6 Insider builds. Apparently a low priority, and I wouldn't be surprised if fewer than 1% of users opted for fully transparent taskbars. FWLIW, I do so in a secondary account for no better reason than my Chromebook is set up with a transparent shelf.

  10. lvthunder

    I'm guessing the Calendar app team is not very big. If I had my guess I would say that number is 1. And they probably work on more then just the calendar app. Just like in most jobs it just probably boils down to where it sits in that person's priorities.

    • wright_is

      In reply to lvthunder:

      Exactly, a lot of people won't use calendar and will use an alternative or the web. I have an account, but I've always used Outlook (fat client), I've never used the inbuilt calendar/people/mail apps.

      • hrlngrv

        In reply to wright_is:

        FWIW, my wife and one child are adamant Apple users, another child is transitioning from Apple to Android/Windows, the third child is Android/Linux, and I'm anything but Apple. A shared Google calendar works best for us personally. As for work, Outlook.

        The only time I ever used a bundled calendar was back in Windows 3.x. Then we moved to Lotus Notes at work, and bundled calendars went the way of the 1-horse trap.

        Given the number of online alternatives and services, I figure PWAs should be the primary approach to calendars. Fully local calendar apps do seem rather archaic.

  11. waethorn

    Read this:

  12. minke

    Talking to some insiders it is very hard for us on the outside to judge the relative importance of the problems we see that are impacting us. For example, it might seem that a problem seen by thousands of people is a big deal, when in reality everyone is focused on fixing something that is causing issues for millions of people. Software is so complex that seemingly minor changes can have ripple effects that destroy something else, so there are implications for any change. In other words, a fix of your calendar problem might destroy some other process that is even more important.

    • nkhughes

      In reply to Minke:

      I don't doubt the complexity of maintaining/updating/improving the OS. I maintain a legacy Win32 ERP app and that's bad enough.

      But the fact is, Calendar was able to access iCloud events fine before 1903. And then they broke it and it took months to fix...any longer and they might as well have waited until 1910 (or 19H2, or whatever it's called). So either they didn't test this combination and it caught them out, or they knew it would break and went ahead anyway. Complexity or not, that's pretty bad from a customer service point of view.

      And if you're right and there was a more critical process that they couldn't risk breaking, wouldn't it be great if Microsoft just let us know why we had to put up with this for months. They can be technical with the details if need be. Instead they said nothing.

  13. Winner

    Microsoft outsourced their testing to the free testers.

    We see how that is going.

  14. moruobai

    This exact problem happened to my calendar! It was synced with my iPhone and then suddenly stopped working. I switched over to Outlook's calendar afterward. There's a lot of apologists in the comments but I think this type of programming and production releases are unacceptably bad.