Microsoft Reveals New Supercomputer

Back in July of last year, Microsoft and OpenAI announced a partnership, with Microsoft investing $1 billion in OpenAI. At the time, the companies said they will work together to build new Azure AI supercomputing tech, and looks like that project has finally come into fruition.

At Build 2020, Microsoft just revealed its new supercomputer. The company’s new supercomputer, developed for OpenAI, ranks in the top five supercomputers in the world. And needless to say, there is a ton of power inside the supercomputer.

[ad unit=’in_content_premium_block’]

To be exact, the supercomputer is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second network connectivity for each of the GPU servers. I will let that sink in for a second.

Microsoft’s new supercomputer was built to tackle massive AI models. Traditionally, researchers built separate, smaller AI models to train AI models, but the new class of models involves creating a single, massive AI model that can better understand language in NLP (Natural Langauge Processing). “This type of model can so deeply absorb the nuances of language, grammar, knowledge, concepts and context that it can excel at multiple tasks: summarizing a lengthy speech, moderating content in live gaming chats, finding relevant passages across thousands of legal files or even generating code from scouring GitHub,” said Microsoft.

Training such a model obviously requires a ton of power, and as part of its AI at Scale initiative, Microsoft built this new supercomputer for OpenAI. “As we’ve learned more and more about what we need and the different limits of all the components that make up a supercomputer, we were really able to say, ‘If we could design our dream system, what would it look like?’” said Sam Altman, CEO of OpenAI. said OpenAI CEO.

“And then Microsoft was able to build it,” Altman added.

Microsoft also announced that the company will give developers access to the language models used within the company via Azure AI. “For customers who want to push their AI ambitions but who don’t require a dedicated supercomputer, Azure AI provides access to powerful compute with the same set of AI accelerators and networks that also power the supercomputer,” the company said.

The company is open-sourcing its Microsoft Turing models, along with recipes for training them within Azure Machine Learning. Developers can use these resources to access the same powerful language models that power language understanding within Microsoft apps such as Office and Bing.