Ask Paul: May 26

Happy Friday! We're coming up on a long Memorial Day weekend here in the U.S., so let's kick it off early with some great reader questions.

christianwilson asks:
Energy consumption
The push to integrate AI into everything got me thinking about energy consumption.

We are living in a time where we are told energy efficiency is critical, yet we are now seeing AI, which is currently far less efficient than a traditional Google search, being added to everything. I don’t think it is as bad as the cryptocurrency energy use (but admittedly haven’t done my research to know for sure) but it looks like we are going backward by using non-efficient tools before they are optimized. Sometimes i feel like companies and end users alike opt to drop their stance on climate/resource control for the sake of convenience. But maybe AI isn’t as terribly inefficient as I am led to believe. Do you have any thoughts on this?

I don't know a lot about this, so here are a few observations.

Whatever the number is, AI queries cost some multiple of normal search queries, so, yes, one can assume that a big chunk of that comes in the form of energy consumption. On the flip side, one of the big pushes right now is in designing more efficient datacenter chipsets to make this kind of thing viable and more affordable. (As Microsoft is doing with Project Athena.) And on the client side, we're moving from GPUs to NPUs for the same reason, and the resulting hybrid AI solutions should also help with that too, by offloading as much as possible from the cloud. I guess what I'm trying to say in my own uneducated way is that this should get better after the initial surge. We'll see.

I don't think we wrote about this, but Microsoft this month literally invested in a nuclear fusion startup, which reads like an Onion headline. This is about reaching its goal to be carbon negative by 2030, which seems like an impossible goal given all this AI stuff. But this is the company that sank a datacenter to the bottom of the ocean in a bid to see whether they could cool it efficiently. One wonders what a lot of those would do to the oceans and the earth's climate, of course, but these things are always a problem for some future generation. Which is a problem.

I just read in The Washington Post this morning that NVIDIA is "suddenly one of the most valuable companies in the world." Why? Because Big Tech companies are using their GPUs to run AI workloads. This company is about to surpass a $1 trillion market capitalization, and it's now the sixth-biggest company in the world. I still think of NVIDIA as a company that makes graphics cards for PCs, but for reasons it never predicted, those chipsets were/are ideal in the pre-NPU world. It's sort of like describing Nintendo's success in gaming as a continuum from its start as a maker of playing cards. That is, its originators could never have predicted the successes to come.

We live in an area of Pennsylvania that largely con...