Microsoft Mesh is a new mixed-reality platform powered by Azure that allows people in different physical locations to share holographic experiences together.
“This has been the dream for mixed reality, the idea from the very beginning,” Microsoft Technical Fellow Alex Kipman said. “You can actually feel like you’re in the same place with someone sharing content or you can teleport from different mixed reality devices and be present with people even when you’re not physically together.”
Microsoft Mesh was featured during the Ignite 2021 keynote today, with Kipman appearing first in an immersive underwater experience, followed by several other environments. This is kind of odd when you think about it: Microsoft hadn’t heavily promoted its Augmented Reality (AR) or Windows Mixed Reality (WMR) solutions in over two years. But now the firm is (re)affirming that it views mixed-reality (MR) as “the fourth wave in computing [following] mainframes, PCs, and smartphones,” and claims that the technology is somehow going mainstream despite no evidence of that happening.
But here are the numbers it provides: 100’s of millions of people around the world have experienced MR through “the devices in your pocket,” by which it means Pokemon Go on smartphones. And over 50 percent of Fortune 500 companies have “deployed HoloLens and other Mixed Reality solutions,” meaning that Microsoft and its partners have collectively sold at least 250 of those headsets.
Fortunately, Microsoft is a bit more realistic about the challenges that developers face in bringing MR to the masses: Representing people in MR with appropriate realism requires a lot of time and resources, keeping a hologram stable in a shared MR space across time and device types is a non-trivial problem, it is hard to bring high-fidelity 3D models into MR to support the file formats our customers have, and synchronizing actions and expressions of people in a geographically distributed MR session is complex, it says.
These are the challenges that Microsoft intends to solve with Mesh. The platform is based on years of Microsoft research and development in areas it says range from hand and eye-tracking and HoloLens development to creating persistent holograms and artificial intelligence models that can create expressive avatars. Mesh is built on top of Microsoft Azure, and it will provide to developers “a full suite of AI-powered tools for avatars, session management, spatial rendering, synchronization across multiple users and holoportation to build collaborative solutions in mixed reality.”
Most important, perhaps, Mesh isn’t limited to just HoloLens and WMR headsets. Solutions built on this platform will also work on smartphones, tablets, and PCs too.
You can learn more about Microsoft Mesh from the Microsoft website. And developers interested in Mesh should check out this technical overview.