Boom or Bust? Thinking About the AI Hardware Upgrade Cycle (Premium)

Most understand that on-device AI requires a powerful NPU to function properly, but there are other requirements that get less attention, like RAM and storage. And we must face the understandable reality of upsell: Some AI features will be limited to new devices only so that hardware makers can benefit from an exaggerated upgrade cycle.

Will it work?

The non-NPU hardware requirements first came to light when Google brought its Gemini Nano on-device small language model (SML) to the Pixel 8 Pro in late 2023. It explains why Copilot+ PCs have a 16 GB RAM minimum, compared to 4 GB for other Windows 11 PCs, and a 256 GB of (non-HDD) storage minimum, compared to 64 GB (which can be HDD). And now we’re seeing this issue again with Apple Intelligence, which will be backported to the iPhone 15 Pro series but not the non-Pro iPhone 15s.

On its Apple Intelligence page, Apple explains that its hardware-accelerated, hybrid AI system will be made available on all Apple Silicon (M1, M2, M3, M4 series) Macs and iPads, and on the iPhone 15 Pro and Pro Max. But those are the only two iPhones supported: The iPhone 15 and iPhone 15 Plus don’t make the cut-off.

This isn’t artificial, like the Windows 11 hardware requirements. The iPhone 15s fall short in two key areas for on-device AI.

Yes, the first one is the NPU. The base iPhone 15s have a lesser Apple Silicon processor—an A16 Bionic vs. the Pro’s A17 Pro—with a far less powerful NPU that delivers just 17 TOPS of hardware accelerated AI performance. The A17 Pro is twice as fast, at 35 TOPS.

But it’s not just the NPU. The iPhone 15s also don’t have enough RAM to handle on-device AI: 6 GB vs. the 8 GB in the Pro models. Thanks to AI, phone makers will be installing a lot more RAM in their devices than was the case until now.

Storage might also be an issue—the base iPhone 15s can be had with as little as 128 GB of storage compared to the 256 GB minimum on the Pros—but it seems like Apple will use only a handful of on-device models, compared to one on Pixel (see below) and over 40 (!) on Microsoft’s Copilot+ PCs. It’s likely that the processor/NPU and RAM differences are the bigger issue.

And I believe that because of what happened to Google.

When Google announced its Pixel 8 family of phones in late 2023, it heavily promoted a wide range of AI capabilities, as it did with previous Pixels. But that December, the online giant announced its Gemini family of AI models, and that the smallest of those models, Gemini Nano, would be used on-device on the Pixel 8 Pro, thanks to a Pixel Feature Drop.

Though this was the first time a phone maker put a modern SLM on a phone, Gemini Nano is still used for just two AI-accelerated features, as was the case at its launch.

“As the first smartphone engineered for Gemini Nano, the Pixel 8 Pro uses the power of Google Tensor G3 to deliver two expanded features: Summarize in Recorder and Smart Reply in Gboard,” Google explained. “Gemini Nano running on Pixel 8 Pro offers several advantages by design, helping prevent sensitive data from leaving the phone, as well as offering the ability to use features without a network connection.”

Predictably, the Pixel fan base erupted with outrage: Why wouldn’t Google also bring Gemini Nano to the base Pixel 8, which had been released less than two months earlier alongside the Pro? After all, the two phones shared the same Google Tensor 3 processor with the same powerful NPU. (In this way, Google differentiates its base and premium phones a bit differently than Apple.)

Google explained that the base Pixel 8 had “some hardware limitations” that precluded the use of Gemini Nano, but didn’t get more specific. It didn’t need to: In comparing the specifications of the Pixel 8, Pixel 8 Pro, and Samsung Galaxy S24 series (which were also getting on-device Gemini Nano) at the time, I concluded that the issue was RAM (and not storage): The Pixel 8 only has 8 GB of RAM, compared to 12 GB for the Pixel 8 Pro.

But the base Galaxy S24 phone also provides just 8 GB of RAM, you might argue. True. But all S24 phones utilize a Qualcomm Snapdragon 8 Gen 3 processor, which delivers better overall performance and better hardware accelerated AI performance than Google’s Tensor 3. And this is where things get a bit nuanced: Though the performance of each SoC’s NPU are similar—29 TOPs for Qualcomm, 27 TOPS for Google—the Tensor 3’s CPU are GPU are lackluster where Qualcomm’s are next-level. This matters because the Snapdragon 8 Gen 3 has a sophisticated orchestrator called AI Engine that pushes AI workloads to the CPU, GPU, and/or NPU optimally, resulting in dramatically better overall hardware accelerated AI performance. It is the only mobile processor that supports on-device Stable Diffusion, for example.

Still, the Pixel user base was understandably upset about the snub, and in March Google gave in and announced that it would bring Gemini Nano to the base Pixel 8, albeit only as a developer-focused opt-in feature. Since then, Google released the Pixel 8a and delivered a Pixel Feature Drop that (optionally) adds Gemini Nano on-device on the Pixel 8 and 8a too.

Though the Pixel 8 and iPhone 15 dramas have obvious parallels, there is one major difference: Apple sells many, many more phones each year than does Google. And where it’s likely that fewer than one million people were “impacted” by the lack of Gemini Nano on the base Pixel 8—it’s just two features, people—Apple’s inability to bring on-device AI capabilities to the iPhone 15 and 15 Plus will impact many millions of customers. Hundreds of millions, most likely.

It’s a problem, for sure. But also a temporary problem. And, from Apple’s perspective, an opportunity: By restricting its most exciting software advances in years to just the iPhone 15 Pro family, Apple may find itself benefitting from an iPhone upgrade cycle of unprecedented proportions. This could be a perfect storm, and it’s one that will bedevil a Microsoft that’s attempting to jumpstart a similar upgrade cycle with Windows PCs. Where Recall and the other Copilot+ PC AI features have been met with disinterest or even open hostility, Apple’s similar advances arrived to nearly universal applause.

This isn’t just because of Apple’s compliant customer base: I’ve already discussed how Apple’s approach to AI is superior to Microsoft’s, and that it’s not just marketing but rather a mix of being provably trustworthy and making this transition in a calm, non-chaotic way.

Less provable is the line between requirement and upsell.

That is, on-device AI really does require powerful hardware. But we may never know whether every AI feature that Apple, Google, or Microsoft adds to their platforms literally require those components. In some cases, each will arbitrarily limit the use of certain features to certain new (or newish) devices in a bid to make the entire package more desirable and drive hardware upgrades.

We’ll find out eventually: Hackers will bypass the restrictions in each system, and we will no doubt discover that certain AI features on each platform will run just fine—or, will run, if less optimally—on systems that don’t meet the literal requirements. We’ve already seen this with Recall, as it turns out: Those who undermined Microsoft’s launch of this feature also discovered that it runs just fine for the most part on normal (non-Copilot+) PCs. The exception appears to be related to parsing Recall’s snapshot database as it grows in size. At some point, an NPU really does help.

This semi-artificial divide can be frustrating, and one might argue that having an AI feature that runs a bit slowly (or less efficiently) is better than not having it at all. On that note, we can expect third-party utilities of all kinds that will enable AI features on each platform that are officially locked behind their maker’s hardware requirements. But again, we should at least respect that Apple, Google, and Microsoft are businesses and not charities. And that, as businesses, their respective desire to ship more units is both understandable and necessary.

And this kind of differentiation isn’t new. The differences between Windows 11 Home and Pro are literally arbitrary, for example, as is the price that Microsoft charges for each. But as the owner of this platform, Microsoft can choose its go-to-market strategies, and part of that is giving each product edition a specific feature set, with the more expensive offering delivering more value. AI is just the latest example of this type of thing.

But here, again, Apple is unique.

For its part, Apple is often criticized, and correctly, for the strangely low-end base storage (and, where possible, RAM) allotments it provides on its devices and for how expensive upgrades are. Worse, with the move to Apple Silicon, these configurations are hard-coded: Once you purchase a Mac, an iPad, or an iPhone, you get what you get, forever, as there are no RAM or storage upgrades available after the initial configuration is purchased.

One might argue that Apple’s purchase-time upgrade strategy, which has been incredibly lucrative to the company, is now coming back to haunt it. That, had Apple not cheaped-out on RAM and storage in the iPhone 15 and 15 Plus, it might have had a much bigger customer base in place to use the new Apple Intelligence features this fall. But again, this is an opportunity for Apple, not a challenge. And maybe I’m missing something, but I’ve not seen any complaints about the iPhone 15 Pro requirement. Where the tiny Pixel community threw a hissy fit over nothing, the gigantic iPhone community has reacted with a shrug. Many are likely planning their next iPhone purchase this coming September.

In this, Apple is like the Tom Brady-era New England Patriots: Feared, admired, and hated at the same time, and the subject of deep jealousies and suspicions. Its competitors never stop trying to copy it while wondering why they, too, can’t ever gain that same level of success. But Apple, like that sports team, is also unique, and imitation is rarely if ever successful. Google and Microsoft may wish to benefit from a generational hardware upgrade bonanza, and they are certainly doing what they can to make it happen. But in raw numbers, I suspect that Apple’s customer base will embrace its AI and the new hardware it requires in numbers that dwarf what we see elsewhere.

In any event, the upgrade cycles will happen, and at whatever respective paces. Phones, tablets, and PCs with hardware accelerated AI features will offer advantages over their suddenly pedestrian predecessors. And when customers do upgrade, the base of AI-powered devices will grow and grow. In time, all mainstream personal computing devices will be powered by SoCs with powerful integrated CPUs, GPUs, and NPUs, and with a lot more RAM and storage than is the case today.

This AI era is unfolding at such an accelerated rate that it’s natural to wonder when it will slow down. This hardware upgrade requirement will be an interesting test. PC upgrade cycles slowed dramatically many years ago, and phones and tablets upgrade cycles have fallen in line in more recent years. Whether the promise or reality of AI will be enough to overcome these trends remains to be seen. And though analysts are bullish, I’m a bit less sanguine: While I believe, as noted above, that Apple will see the most growth in this area, nothing is certain. The AI boom could turn into an AI bust if consumers and businesses don’t fall for the hype.

Of course, the AI bust is coming regardless. Until then, it’s fascinating watching each platform, hardware, and software maker tripping over each other to line up to shovel AI down our throats, like thirsty animals at a dwindling, dried up oasis. You can almost sense the greedy desire to sell something that each, in its own way, is unsure of. Some of it will land, much of it will not. And we will emerge on the other side of this hype cycle, this upgrade cycle, changed. As will the devices we use every day.

It’s a wonderful and scary time to be alive.