Windows Copilot Runtime Isn’t a Runtime, and WPF is Back, Baby (Premium)

During Monday’s Microsoft AI event, Pavan Davuluri threw out an unexpected term, Windows Copilot runtime, without really explaining it. Today, we have a bit more information about that, and some news about the new approach to “native” apps on Windows.

“We have infused AI into every layer of Windows, including a fundamental transformation of the OS itself to enable developers to accelerate AI development on Windows,” Mr. Davuluri writes in a new Windows Developer Blog post today. “Windows Copilot Runtime has everything you need to build great AI experiences regardless of where you are on your AI journey – whether you are just getting started or already have your own models.”

The Windows Copilot Runtime doesn’t appear to be a runtime, despite the name (like .NET). Instead, it’s better described as a platform that encompasses an API called the Windows Copilot Library, a set of AI frameworks (DirectML, ONNX Runtime, PyTorch, and WebNN), and so-called toolchains like Olive and AI Toolkit for Visual Studio Code.

Given my years-long attempts at understanding the future of Windows app development, I’m most interested in the Windows Copilot Library. This is described as a set of APIs powered by the 40+ on-device AI models that ship with Windows, including the new Phi-Silica model that was created specifically for Copilot+ PCs. (That 40+ number is rather impressive and unexpected.) These APIs work across the Windows shell, Win32 (desktop, but really WPF/NET presumably), and web apps, Microsoft says.

Interesting. But also perhaps not entirely accurate.

During the Build 2024 keynote today, Microsoft CEO Satya Nadella compared it to the Win32 API, and that’s technically true in that it’s a set of APIs. But without being able to see these APIs now—it’s coming in June—I predict that none of this is literally native code, and that these libraries are higher-level abstractions that target .NET and web only.

From a functional perspective, the Windows Copilot Library will initially give developers APIs for Studio Effects, Live Captions Translations, OCR, Recall with User Activity, and Phi Silica. Then, a later release will add APIs for Vector Embeddings, Retrieval Augmented Generation (RAG), Text Summarization, and more.

It’s not clear what Microsoft means by this, but the Windows Copilot Library will also give developers “no code effort” access to Studio

Effects like Creative filters, Portrait light, Eye contact teleprompter, Portrait blur, and Voice focus. That doesn’t mean “no code,” but … it’s unclear. Access to Live Caption’s real-time translation capabilities is similarly a “no code effort.”

The Windows Copilot Library will also help developers integrate their apps with the Recall feature that’s exclusive to Copilot+ PCs. This support includes User Activity API that provides access to the underlying underlying vector database used by Recall. Microsoft apps like Outlook, PowerPoint, and Teams are using this API to provide Recall compatibility.

And here’s some news for those concerned about “native” Windows apps: Microsoft has not subtly shifted the emphasis back—way back—to Windows Presentation Foundation (WPF), a technology that debuted in 2006 in Windows Vista, and WinUI 3, the modern user interface framework. This hybrid Frankenstein monster of the new and old is in no way native when used together, but both can take advantage of modern features and APIs in the Windows App SDK.

Speaking of which, Windows App SDK has been updated to version 1.5 and it now supports .NET 8, a WebView 2-powered Maps control, and other new features. Microsoft notes that the Windows in-box apps Photos and File Explorer have “migrated” to WinUI 3, while third-party developers like Apple—with Apple TV, Apple Music, iCloud, Apple Devices—are using this technology as well to deliver modern apps through the Microsoft Store.

As for WPF, this weary veteran of the Mac v. PC wars of 20 years ago is still “popular,” Microsoft finally admits, “especially for data-heavy and enterprise apps.” Listening to feedback for a change, the software giant says it is now “committed to continuing investments in WPF.” It has updated WPF recently with Windows 11 integration support for such features as OS themes, folder dialogs, and managed DWrite.

What’s old is new again.

Put another way, the future of native apps on Windows is hybrid apps on Windows.

But the future of apps on Windows, more broadly, hasn’t changed. It’s still web apps. More on that as soon as I can find the time.