Microsoft is AI-In on Hardware

Microsoft is ramping up its use of chipsets that are optimized for AI, both on the backend and in its coming generation of Surface computers. For its datacenters, Microsoft is designing its own AI chipsets in a bid to reduce the heady costs that come from using large language models. And for its PCs, the software giant will rely on neural processing units (NPUs) provided by AMD, Intel, and Qualcomm.

Let’s start with the backend: here, The Information reports that Microsoft has been working on an AI chipset, codenamed Athena, since at least 2019, and that it has sped up this effort as it moves to add AI capabilities across its product lines. The firm is already testing the chipset with a small group of Microsoft and OpenAI employees, and the hope is that it will perform better than third-party chipsets, saving it significant time and money. (As the publication notes, Amazon, Google, and Meta all make their own in-house chips for AI already.)

“Microsoft wants to use large-language models across all their applications including Bing, [Microsoft] 365, and GitHub,” SemiAnalysis chief analyst Dylan Patel told The Information. “To deploy that at scale using off-the-shelf hardware would cost tens of billions of dollars a year.”

There are over 300 people working on Athena and Microsoft expects the AI chipset to be ready by next year, though it’s not clear yet whether it will make it available to Azure customers. It’s likewise not clear what this means to Microsoft’s supercomputer partnership with Nvidia.

Separately, I’ve been told by two sources that Microsoft this week disclosed to its MVPs that its next-generation Surface PCs would include NPUs to speed up AI and machine learning (ML) on the devices. Microsoft currently uses microprocessors from AMD, Intel, and Qualcomm, and each has or will be adding NPUs to their chipsets. (The Qualcomm-based version of Surface Pro 9 is the first Surface PC to include an NPU.)