Ask Paul: May 31 (Premium)

Happy Friday! It’s been a busy few weeks, but I’m not surprised that many of you are thinking about the same things as I am. So let’s jump in and kick off the weekend a bit early with some terrific questions.

AI PC, more like DOA PC

A couple of related questions about AI PCs…

madthinus asks:

Between Ai PC and Copilot+ PC arriving less than a year apart, both proclaimed to be the next thing, is the 40 TOPs arbitrary like 8gen Chips was for Windows 11 or are they waking up to the reality of on Chip AI?

40 TOPS doesn’t feel arbitrary to me. And while this “session” isn’t as on-point as his Build 2023 appearance, Stevie Bathiche gave a very short talk at Build 2024 that neatly explains Microsoft’s focus on the NPU (among other things).

We all know that GPUs can handle graphical workloads more efficiently than CPUs, and that using one can thus provide this magical combination of better performance and battery life. But the way Stevie describes these different processors is fascinating to me. CPUs are optimal for scalars, like numbers. GPUs are optimal for vectors, like arrays of numbers. But NPUs are optimal for tensors, like arrays of arrays.

“NPUs … are basically tensor accelerators,” he says. “NPUs are purpose built to handle tensors. All the way from the entire Silicon architecture to the software stack is all about managing this data type. It gives us a tremendous amount of efficiency as a result.”

I try to explain things in simple terms and AI is particularly challenging because there’s a lot of new and unfamiliar terminology, and because it’s, well, complicated computer science. So I typically boil down the NPU to something like “hardware accelerated AI.” But Stevie gives a great real-world example of how and why the NPU is so important for generative AI tasks by visually showing how much more efficient it is than a CPU or GPU. It’s dramatic.

But the basics still apply. In the same way that an GPU can offload tasks from the CPU and improve performance and battery life, the NPU can perform certain tasks in the background so efficiently that the net impact to the system is negligible. To date, the big example of this type of task is Windows Studio Effects: You’re on a video call for 30, 60 minutes, whatever, and it’s in the background doing its thing, blurring or changing your surroundings, killing background noise, and so on, and there’s basically zero impact on system performance or battery life. The CPU and GPU could both handle that work, but harming performance and battery life. GPUs can have higher TOPS scores than NPUs, but they are not even close on efficiency. Which is why NPUs are so important on Edge devices like laptops.

Windows Studio Effects works fine on a 7 TOPs AI PC. But to do the work that Microsoft showed off recently, like Recall, which runs in the background and uses multiple on-device SLMs simultaneously, raw performance isn’t enough. Efficiency matters.

“The way we built Recall was having these models run consistently and constantly in the background to detect your region, do OCR, do image encoding, text encoding, and a natural language parser, all running in the background without burning up your battery life,” he says.

And 40 TOPS is, in Stevie’s words, “The minimum bar to actually run all these experiences.” He does a demo in which all 40+ on-device SLMs are running simultaneously, and the GPU isn’t impacted at all.

“While I’m running this entire workload, I can run a game,” he says. “And that’s pretty awesome. That’s the type of power that sits in these Copilot+ PCs. This is a laptop, for crying out loud, doing this workload.”

Again, this talk isn’t quite as effective as last year’s appearance. But if you care about, doubt, or are wondering about the 40 TOPS NPU thing, you need to watch it.

Related to this, wright_is asks:

Last summer, Intel and Microsoft announced that all previous generations of Core processors were obsolete and you needed an Ultra processor to have a “new” AI PC, so anybody who bought a new PC the week before was now the proud owner of a computing dinosaur…

So.

Intel and Microsoft didn’t announce anything about obsoleting previous generation Core processors. Intel belated added an NPU to a mainstream processor family, decided to call it something different, and delivered on the minimum experience required to achieve the only AI accelerated feature in Windows at the time, Studio Effects. It subsequently released a new generation of (non-Ultra) Core processors and I gotta be real here, I put the blame for all of this, the confusion and the terrible products, on Intel, not Microsoft.

We see a lot of this kind of confusion in the hardware space. USB ports are a great example: Instead of just giving every PC the highest-end ports (Thunderbolt 4/USB4/Type-C), PC makers deliver a confusing mix of ports, forcing consumers to figure out which does what. Why? The excuse is always penny-pinching, but I noticed something interesting when I wrote about the new Framework Laptop yesterday: Each type of USB expansion card they sell costs the same amount of money, $9. USB-C, USB-A, doesn’t matter. It’s $9 and you get the past available part every time (Thunderbolt 4 for Type-C and USB 3.2 Gen 2 for Type-A.) And you know big PC makers get these parts for nothing. They’re probably saving sub-pennies by not just doing the right thing.

In Intel’s case, Microsoft spent approximately two decades begging this company to deliver more efficient chips, and Intel kept shooting for high-performance and low efficiency. And so this mess is of Intel’s making. It had the engineering expertise to do what it’s slowly doing now long ago, it just didn’t do, sacrificing long-term viability for short-term profits on higher-end, more expensive (but less efficient) parts while the rest of the industry moved forward without it. So Intel will belatedly deliver a 40 TOPs NPU in its second-generation Core Ultra chips this fall, right on its own stupidly slow schedule. And in time for Windows 11 24H2. This is just a weird 6-month gap with Qualcomm in the market first.

Could Intel have delivered that 40 TOPS NPU in a new chipset before this fall? Obviously. But it has its own schedule, and as is the case with all partnerships, each company has their own priorities. And if we know Intel like we all know Intel, we all know it’s going to deliver more non-NPU-based CPUs in the months/years to come, too.

Anyway.

Skip forward 6-7 months and Intel and Microsoft announced that anyone silly enough to have bought a new AI PC over the last 6 months is now the proud owner of a computing dinosaur, because they won’t be capable of running Windows AI features, because the highly prized NPUs that they lauded last fall are now way to slow to do anything useful and you will need to wait until this fall for new Intel processors, unless you want to buy a Qualcomm Snapdragon X Elite PC…

Here’s how I frame this: Intel belatedly discovers NPUs, a feature Apple and Qualcomm had been adding to their SoCs for years, and invents yet another specification, because that’s what they do–Intel Evo, anyone?–so that PC makers can put yet another stupid sticker on their hardware. And in doing so, they deliver the minimum, to address yesterday’s “problem,” which is just having a NPU at all. As I write this, all the AI capabilities in Windows 11 in stable, save Windows Studio Effects, work in software and don’t require an NPU. And that’s true of Windows 11 24H2 too.

What changed is that Microsoft has been working to figure out a higher-end set of AI capabilities, which they just announced, that require more powerful NPUs. Intel could have just freaking waited for this. But they went early with a lackluster NPU in Core Ultra v1. So this confusion is almost certainly all on them: You have to know that Microsoft fought to prevent this from happening and Intel just had different priorities.

Here’s what’s interesting. If you look up the PC maker announcements about their Copilot+ PCs, you will consistently see them all refer to these devices as “AI PCs” because that is the Intel spec and Intel pays them to use that. But PC makers also like to have premium, high margin products, and the Copilot+ PC “spec,” which is a superset of AI PC, is also desirable. So they use both where they can, as with these newly announced PCs. This is confusing and stupid. And typical.

(In its announcement, Dell uses the term “AI PC” 11 times and “Copilot+ PC” just 5 times; the term “Copilot+” appears 10 times. Lenovo uses “AI PC” 3 times and “Copilot+ PC” 6 times. HP uses “AI PC” 27 times and “Copilot+ PC” just 1 time. Guess who’s in bed with whom?)

To summarize, AI PC and Copilot+ PC are “specifications.” Windows 11 has a growing set of AI capabilities, only one of which requires an NPU. And Copilot+ is a set of AI experiences that requires specific hardware–NPU with 40 TOPS–that’s only available when you buy a new Copilot+ PC.

That’s an explanation, not an excuse. I think it’s all stupid.

We are used to processor generations being faster and offering some new features, but I can’t remember Intel & Microsoft ever stabbing their customers in the back like this, especially when the market is unstable and declining at the moment, surely Intel needs as many customers at it can get, deliberately alienating them, twice, in one year doesn’t sound like a good idea!

I’m projecting here, but I think we’re seeing the AI version of a land grab, where we suddenly have this exciting technology that everyone is trying to capitalize on, and to do so, they are hurting their own partners and their own customers to make it happen in a way that is most advantageous for them. And on that note, we have a word for that. Enshittification.

So there you go.

Get off your NAS

MichaelMDiv asks:

Hey Paul, has Windows 11 24H2 release changed your ability to sync with or use your NAS? I think you are still using the WD, which uses SMB 1.0/1.1, and I saw a MS employee quoted saying they have killed that technology/connection. I was skimming, so I might have missed important details, and you haven’t mentioned any issues, but wanted to ask about your experience with this.

I’m traveling and will be away from home until the end of June aside from this coming Monday, so I’ve not tried to hit the NAS since I installed Windows 11 version 24H2 on most of my PCs. But this is a known problem, which Microsoft addresses on its Tech Community site. The issue isn’t SMB 1.0 per se–you can still enable that feature in 24H2–it’s SMB signing, which is required by default on all connections. That post has a list of workarounds/fixes for getting that connection working again.

.NETpad on other platforms

spacecamel asks:

Since you have the MacBook Air, have you considered writing a version of your application in Swift using X code? Or make a Windows version using Swift similar to what Arc is doing?

I’ve vaguely considered all kinds of ports, including .NET MAUI, Swift/SwiftUI, Flutter, and React Native/web, but these things are time-consuming and the further we push into the future, the less inclined I am to try anything locked to a specific platform. Of these things, React Native is the most interesting to me, and a recent Microsoft Edge team blog post about a modern web text-editing capabilities triggered a bit of research on my part. This is always in the back of my mind.

But now that WPF is officially back, I think that will be my .NETpad-related focus this year. Modernizing an existing app like this is a classic developer challenge, so that seems like a good use of time. But I would also like this to turn into an excuse to finally experiment more with AI from a developer perspective, using both on-device and cloud-based language models. So it could also turn into a little AI experiment too. I can’t imagine having more time this year for anything else developer related.

Must-watch TV

j5 asks:

Hey Paul, “have you seen any good movies or shows lately” ?. But seriously what have you been watching? A few movies and shows I’ve seen … Do you have any recommendations? What’s on your watchlist? Thanks!

Thanks for the recommendations. This kind of thing is frustrating for my wife and me as we always struggle to find new shows/movies that are any good and then forget what we have watched.

Here’s what I can remember.

On Netflix, the most recent show we’ve watched is Bodkin on Netflix, and we both liked it quite a bit. (It’s set in Ireland and is a sort of crime drama/comedy.) Also, Baby Reindeer, which was good but quite disturbing (basically about a stalker and other predators, and based on a true story). Griselda, which could have been part of the Narcos series, and very good. And the final season of The Crown.

We watch a 30-minute (really 22-minute-ish) sitcom-type show at lunch each day, and we’ve been working our way through Modern Family on Hulu, which is very good but wears out its welcome by the last few seasons, like many shows. But we had never seen it before.

Movie-wise, we just rewatched Gone Girl and The Girl on the Train, and in both cases were surprised by how little we remembered each. The second of those is better; Gone Girl ends weirdly and it felt unsatisfying. We also rewatched the first two Lethal Weapon movies, which are OK but dated. Lost in Translation, which we loved. The Big Chill, which was just OK. Late Night with the Devil was quite good, unexpectedly.

I have rewatched a few movies by myself recently, sort of in keeping with that whole rewatchable thing: Prometheus and Alien: Covenant, Heat, and The Russia House most recently. I watched the Road House remake by myself, that was terrible.

I’m sure this is partial. It’s difficult to keep track of this.

Tablets

eeisner asks:

Outside of the iPad Pro, Microsoft really seems to be one of the few still pushing the detachable, 2-in-1 formfactor. Why do you think OEMs have seemingly given up on the form factor?

They don’t sell well enough. Tablets are tough: There are clear use cases in the consumption space, but that’s pretty much just Apple (iPad, iPad Air, iPad mini) and Samsung. The hybrid tablets (Surface Pro and iPad Air, I guess) are a tougher sell for all kinds of reasons. But I have this saying, “optimize for the everyday,” and convertible PCs–Spectre x360, Yoga, etc.–address that nicely, and do sell well: I’ve been told repeatedly that most buyers of these PCs just use them like laptops all the time. But customers get upsold for the “just in case” thing that we’re so susceptible to. So convertibles have replaced tablets/2-in-1s in the PC space for the most part. They work great as laptops.

The Android tablet ecosystem is a pretty strong failure. Why do you think the Android tablet app market never kept up and made Android tablets a more usable product?

Google has done a reasonable job of improving Android to work on big screens, but a few things have held back the ecosystem. One is Google’s fault: It can’t seem to decide whether it wants to back Android, Chromebook, or folding display phones in this space, and so it’s sort of done all three. But the bigger issue, I think, is just that developers in the Apple space seem willing to follow Apple’s lead at all times, whereas Android developers, for whatever reason, are less likely to do so. So we end up with a lot of stretched out phone apps on big screens. This has gotten better, to be clear. But where you get this nice, optimized app experience on iPad, you never know what you’re going to get on Android. Some apps look/works great. Some don’t.

Likewise, Windows tablets never had a strong case after the failure that was iPad envy with Windows 8. Do you think that the battery life, minimal fan usage, and (future) 5g promise of Arm is what Microsoft needs to make the Surface Pro a more productive iPad Pro competitor or entice OEMs to pursue more Windows tablet or detachable devices?

No, because the ecosystem isn’t there: Even the mobile (UWP) apps in the Microsoft Store are mostly oriented for traditional keyboard/mouse usage, not touch. And the world has moved on. There’s no v2 for that world on the horizon. In the “optimize for the everyday” sense, Windows apps are desktop-first and, more often than not, cross-platform anyway.

I have been trying to make the 2-in-1 work in my daily life for a decade now. In college I owned the original Surface RT and the original ASUS Transformer Android tablet at separate points for note taking, a Surface Book at my first job as a gift to myself, and now I have a Samsung Tab with a detachable keyboard for travel. I am looking to replace my aging Surface Laptop 2 with the new Snapdragon Surface Pro. I know you purchased a new Surface Laptop to have a more accurate head-to-head comparison with the Macbook Air, and I know it can be hard to recommend Surface products for a variety of reasons. That being said, do you believe that the Surface Pro is a smart purchase in 2024?

Yes.

Surface has always been a bit of contradiction for me. That is, I have a hard time recommending Surface to people for a variety of reasons, key among them the reliability/quality issues that often pop up. But also, I’ve not reviewed a Surface PC in a couple of years now, and my personal experience is out of date. But my daughter uses a Surface Pro 8 (that I reviewed in late 2021) and loves it. And while many complain about the bounciness of the Type Cover-based keyboard, I’ve never had any issues with that. My experiences with Surface Pro have been mostly positive (since Surface Pro 3 anyway).

But as I get older, I prefer larger displays. And I wanted to get something I’d actually use every day. So that meant Surface Laptop to me, in part because it’s a lot like the MacBook Air, but also because I know I’ll get HP and Lenovo laptops to review, and I want that wider set of experiences. And I have always had a soft spot for Surface Laptop, though I’ve not reviewed one since Surface Laptop 2 and have never used a 15-inch Surface Laptop.

I will say, I could save a lot of money by choosing a different Copilot+ PC: The Surface Laptop I preordered costs over $2200 all-in. An identically configured HP OmniBook X–with 32 GB of RAM and 1 TB of SSD storage–costs just $1400. And since HP offers more configuration options, I could save even more money by getting the 512 GB of SSD I really want, for a total of just $1300. That’s almost a $1000 difference.

But Surface falls into that weird category of decision-making that’s more about emotion than logic. It’s possible that logic will win out in the end: Perhaps I’ll get an HP for review while I’m in Mexico, fall in love with it, cancel my Surface, and just buy one of those. Anything is possible. But to your point: A 2024 Surface Pro, which has a 13-inch display and does have that 2-in-1 versatility, is absolutely a viable upgrade for you, assuming that’s what you want from a form factor perspective. A Surface Laptop 7th generation, ditto. Choosing Surface is partially emotional. But choosing between those models is just a normal, logical decision. Just get what you think will best serve your needs.

And if you need that final push, not that I’m trying to sell you on some specific direction, Microsoft has a more than reasonable return policy: You have a full 60 days to figure out if it’s right for you. I can’t explicitly recommend a Surface, I guess. But assuming my purchase goes through as I expect, I’ll have more data soon.

Windows 11 8 12, wait what

ianceicys asks:

Paul, AI seemingly came out of the blue in ~November 2022, and now with Copilot+PCs in May 2023–seemingly as reaction to AI -it seems like any BIG change at Microsoft takes 1 year to 18 months, so with Panos Panay leaving in September 2023 due to “Budget and Product Cuts”, what are your thoughts on the future a year from now May 2025, at next year’s Build, do you think we’ll still be on Windows 11, or will there be a different path with say Apple coming to complete and totally dominate AI?

The future is difficult to predict in normal times, but AI makes this even more complicated. We’ll learn at WWDC whether Apple will actually make iPad Pro a pro-level device, for example, which for many is a lot less interesting than the obvious AI stuff they’ll announce. But I don’t think Apple dominates AI, I don’t think any company does. AI is coming for us all, everywhere, and it will just be … everywhere. And that poses some challenges, among other things, to companies like Microsoft and Google that want to charge for these capabilities. So this will evolve.

I didn’t predict any of the specifics of what we learned at Build this past month, or even try to. But the broad strokes were obvious enough to see in advance: Microsoft would reveal local, on-device capabilities for Windows at the very least, as it did. And then we would get a version of Windows that offers these features (which it only sort-of did, since most of this stuff is Copilot+ PC-specific).

This is eerily reminiscent of the early 2000s, when Microsoft was bifurcating Windows with Media Center and Tablet PC editions of Windows while Apple was selling just a single version of Mac OS X. And so that sort of thing might repeat here, with Apple just doing AI on its devices and Microsoft trying to upsell customers for the best stuff. Or it may not: Maybe Apple adds some key AI capabilities to services and subscriptions only. What it does will in some ways determine how things evolve. (Google has, to date, just done exactly what Microsoft does with free and paid AI options.)

But a year from now, yes, I expect we’re still talking about Windows 11. Windows 10 will be careening towards end of life, and making things even more complicated with a Windows 12 (or whatever) on top of that is … well, they’re sort of doing that already, right? You might view Copilot+ PC as Windows 12. A way for Microsoft to get paid for things it’s adding to Windows rather than just giving them to everyone using Windows 11 for free. So that’s the bifurcation I see us moving forward with. Until those features are just folded into mainstream Windows when enough PCs out there are powerful enough to use those features.

A year from now, we’ll have much more choice in the Copilot+ PC space, with AMD and Intel-based options. We’ll probably have, or know about, other chip-making partners that will enter the Arm space and try to give Qualcomm a run for its money. (AMD might be among them. Nvidia. Samsung. We’ll see.) Cloud-hosted AI will be both more powerful and less expensive. Hybrid AI will be common. New apps and services will appear. Existing apps and services will be updated with new AI capabilities. And this will all evolve in tandem.

I will say, it’s nice to have something exciting to discuss in the PC space for a change. The Windows 11 story, to date, has been curiously negative, which is depressing. And maybe this is the start of a change for the better. We’ll see.

It’s all coming together

jrzoomer asks:

Paul what makes you excited about the upcoming Snapdragon X Elite in a way that the previous Snapdragon 8cx processor on your Thinkpad 13s didn’t?

These are completely different experiences. The ThinkPad 13s is notable because it was the only major PC to utilize that processor, as most PC makers had either given up entirely or were just waiting on a chipset, the Snapdragon X, that actually worked. But now that it’s arrived, we see something entirely different: Not just technical improvements to performance, emulation, and so on, but the entire industry lining up behind it. This never happened before. Even Microsoft, which was early to Arm support in Surface, has taken it mainstream with two PCs that are now a primary focus.

The ThinkPad 13s was OK for what it was. But it was just a single, tiny, ultraportable PC. (It was fanless and silent, by the way.) It was very expensive, with prices starting at $1810, almost double the cost of a Snapdragon X-based PC. It was more an experiment and an outlier than a viable option. The new PCs are just PCs. Aside from a few esoteric issues some people will have with random hardware peripherals, most will simply buy one and it works.

Another related question, if the AI features have you excited, would it be better to have one with a discrete NVIDIA GPU (which should far exceed the 40 TOPS mandated for Copilot branding)?

No, and this is explained up top: An NPU with 40 TOPS of performance is, if not quite exponentially more efficient than a similar GPU, at least dramatically more efficient. That’s why the NPU makes sense: It does its thing without killing the battery.

Other content services

helix2301 asks:

Where do you stand on services when it pertains to content? I know you have subs to Netflix and others but what about books? Do you sub to any of those like Kindle Unlimited services or local library digital book service?

I don’t subscribe to Kindle Unlimited. But I do subscribe to Audible Premium Plus, which gets me 1 credit towards an audiobook each month. And I have started using the Libby app to connect with my local library, so I can read books, magazine, newspapers, and more for free.

Kindle

helix2301 asks:

Where do you stand on ereaders? I know you use an iPad but I know Leo, Alex, Mika, and others are big into ereaders. I was just wondering why you have never been a fan. I know you are a big audible guy which is an amazing service. Just wondering you thoughts on book services and ereaders.

As a life-long reader, I am the original Kindle fan. I’ve owned almost every Kindle e-reader ever made, and the box from the OG Kindle is behind me on the shelf, which you could see in podcasts if my big head wasn’t in the way (when I’m home, at least). Actually, it’s not clear what it is now that I’m looking at it. (I’m not home right now, so I can’t just take a picture of it.)

But yeah, I moved to the iPad for reading some time ago, and I mostly read my Kindle content way. Oddly, I did recently purchase a new Kindle Paperwhite because I wanted a USB-C-based Kindle, and because it had been a while. I briefly consider the Kindle Scribe because of the larger display, it’s the one major Kindle I’ve never owned. But it’s expensive, I don’t care about the note-taking capabilities at all, and I’m still hoping Amazon will put out a color e-reader someday.

Anyway. I do prefer the Kindle for all kinds of reasons for reading, it’s just that it’s limited, and having the one device (iPad) that handles all reading is more convenient: I read using a variety of apps, not just books in Kindle. Carting around yet another thing doesn’t make a lot of sense. (I did just read Stephen King’s Desperation almost entirely on the Kindle, for whatever reason, it’s a book I had started but never finished previously.)

I love the Kindle. I wish it made sense to use it more often.