Why has so much of the software industry stopped doing QA?


In the past few years, a lot of software/video game studios have decided to just stop doing QA. Microsoft famously fired almost all of the Windows QA people in 2014 or 2015 I believe, now we’re seeing video game studios like CD Projekt RED and Rockstar decided to stop doing basic QA on their games. There are more examples of this trend. But why have companies decided QA isn’t necessary anymore? It ends up hurting the companies reputation and their wallet too, so what’s behind this?

Comments (22)

22 responses to “Why has so much of the software industry stopped doing QA?”

  1. usman

    I guess the proliferation of high-speed internet has allowed games and other software to be patched after it has shipped. This kind of attitude is also adopted by businesses people who then use it to rush things out the door to maximise shareholder value.

    It's not the right attitude to have but the shareholders rather care about something launching on time to meet their forecasted financials rather than testing and dealing which adds more cost and will negatively affect their bottom line.

    The short of it really is corporate greed and disrespect for the engineers that are forced to crunch to meet a deadline and the end consumers who have given their money for a terrible end product.

    It feels quite scammy, that businesses are continuously incentivised to put any form of revenue forward even if it means getting the negative backlash of a poor unfinished product.

    • wright_is

      Testing and QA takes time and money. Having a proper testing team is something that a company sees no real return on, in terms of immediate cashflow, it is a sink hole of time and money...

      In reality, it actually adds more to the company's reputation and goodwill, but goodwill is a long-term accounting item and is totally useless at generating revenue in the next quarter.

      Sending out unfinished software early to an unsuspecting public and getting them to test it for you is a much better, short term, policy. It makes people hate your product, but, hey, they've already bought it and you can throw some update bones at them to try and keep them happy, once the thrall of the new shiny shiny has quickly worn off and the reality that they have bought chrome-plated dog poo starts to dawn on them.

      Real quality software costs real amounts of money. I used to work for a software company that provided software for the food industry. A 15 minute stoppage would soon cost three figures in lost product, so the software had to work non-stop throughout the shift(s). That meant it had to be thoroughly tested, but that also made it expensive. But if you are looking at hundreds of thousands of Euros in waste product that can't be used, if your systems don't work (it was full automation of the production lines, conveyor systems, all integrated into the ERP system), the higher cost of tested software is going to be worth it.

      A copy of Word crashing every few hours? A game that renders certain scenes incorrectly or the player can drop through the floor of the world? Not so much of a problem.

      • willr

        Great reply but a lot of the issues are far more serious than occasional bugs. Windows 10 had some crazy bugs when it first came out, especially when run on Skylake. And I can tell you from first hand experience, Cyberpunk was actually unplayable for the first 4 to 6 months on the Xbox One X, and it probably is still unplayable on the base Xbox One/PS4. And the GTA trilogy that was just released is actually unplayable on the Nintendo Switch, they knew this and released it anyways

  2. lvthunder

    To add on to my thoughts I also think a lot of it has to do with the cost of software. Prices have gone through the floor. Especially for new software in the market. When I got my M1 mac and my M1 iPad I bought Lumafushion which is a really nice video editor. In the past, something like this would be easily in the $100-$200 range. Today it's like $25 and I've gotten new features for free on top of that.

    • jason_e

      This certainly could be some of it. Nobody seems to want to pay a developer for their time and effort anymore. Free, free, free is all anyone wants anymore. Same with things like streaming services. Everyone wants top quality shows but half the people do not want to pay. Hey can I get your login. You are already paying so why should I. Its a victimless crime. I hate that mentality.

  3. beckoningeagle

    We are at fault. The users, that have actually allowed companies to get away with it. Take Google as an example. Half of their production systems were labeled as Beta for years, including GMail, and people still flocked to their services. I can understand in a market like the OS where there is little competition for Microsoft to get away with it, but when it comes to games and other software where there is plenty of competition, we as users have made the bad decisions of allowing this nonsense to continue. Running to download the latest beta versions only to get bit in the ass and then complaining that we lost data.

  4. JackR

    It's been said that any program longer than two lines has a bug in it.

    When upper management and the finance folks learn that their software will ship with bugs, regardless of whether they spend nothing or a zillion bucks on QA; and those bugs will get fixed by 'upgrades', their decision is easy.


  5. bigfire

    I can't speak for any other companies, but most I know have QA or rely on large contracting houses for mass testing. The latter is obviously unrealistic for most companies to do, although I'd argue that the results from these contract houses are poor. For the former, I know that MS used to have a large QA contingent, but later switched over to automated testing for everything. While useful, it's clearly only part of the solution, but MS went in whole hog on it. I know many great QA testers who got let go because they couldn't write automated testing.

    Nowadays (waves cane angrily) it seems like these companies are clapping themselves on the back for making the "smart" economic decision. After all, if we still use MS software, and they're spending less money on testing, one could make the argument to the shareholders that they're doing the right thing. Valve, in particular, seems to relish how successful they are despite how bad Steam is. My counter is that the constant bugs I encounter makes me feel like they don't care about their customers, which creates more of an abusive relationship for the customer. But as long as we put up with it, they won't change.

    I would LOVE to see some analytics from MS that measure bugs in shipping products. I suspect there would be a very noticeable between when they had full QA teams and when they went over to automated testing. Of course, there are a lot of ways to measure meaningful robustness, and I suspect whoever took credit for this terrible idea came up with metrics at the time that showed positive results.

  6. MoopMeep

    You forgot Boeing.

  7. hrlngrv

    Maybe the economics of software as a service makes most sense when there's greater need for service due to chronic need to fix bugs. Or maybe we've simply entered the Era of Perpetual Beta Software. Either way, I figure it's a dead certainty software vendors have determined this is the most efficient way for them to develop and deliver software to maximize their profits.

    If you want to know who's ultimately responsible for this state of affairs, find a mirror. As long as we all keep paying for software with lesser quality than software before, say, 2010, we have no one but ourselves to blame.

    • navarac

      Guilty party here for past participation in the MSFT Insider Program on Windows 10. Sigh!

  8. lvthunder

    But does it hurt their wallet? I would say it doesn't. I also don't think it hurts their reputation to the point that people look elsewhere. People have come to expect bugs in software since they are easy to update and fix.

    The most embarrassing one of these is that Cyberpunk game that they had to pull. Does anyone even know the studio behind that game so they know to avoid the next game they make? I know I don't, but I'm not that big of a gamer.

    • willr

      In CD Projekt RED's case, yes for sure. In Microsoft's case? Probably not but who knows for sure

    • christianwilson

      Yeah, Cyberpunk 2077's developer, CD Projekt RED, earned a positive reputation with gamers because of their work on The Witcher games. They aren't a household name like EA or Activision, but within the game space they are well known and were well-regarded.

      I don't know if Cyberpunk's crash and burn will damage CDPR long term or not. I think it is ok for a developer to aim high and not quite hit the mark, but Cyberpunk's technical issues are spectacularly bad and the company's communication around the game's launch upset a lot of people.

      I will still buy CDPR games once I know they aren't in a broken state but they lost my trust in being an early adopter.

  9. darkgrayknight

    If Test Driven Development is occurring, there is likely the idea that it is already tested (bad idea, but it is there). TDD is beneficial but it certainly doesn't replace actual QA testing.

  10. navarac

    Pure shareholder greed, I reckon. Plus they don't care once they have your hard-earned cash.

    • hrlngrv

      | Pure shareholder greed

      That's something new, is it? World & dog have been taken by surprise by it?

      Or does software bear some uncomfortable similarities to addictive drugs?

      • anoldamigauser

        I would say it is pure C-Suite greed. They are the shareholders that matter, since they are the ones making the decisions and reaping the largest benefit in the form of stock options, which only have value if the stock price is above the strike price, and performance bonuses.

      • navarac

        No - it's nothing new, of course. But in this day and age of social media etc., bad reviews spread like wildfire - about 10 times fast than the good news.

  11. willr

    and The Verge just published an article called "Call of Duty made $3 billion in 2020, so of course Activision Blizzard is laying off QA" ...

    "Raven Software, known for its work on Call of Duty: Black Ops Cold War and CoD: Warzone has now laid off a third of its contract QA testers, according to reports by Kotaku and The Washington Post."