Ask Paul: April 25 (Premium)

Happy Friday! I got off lightly last week, but we're making up for that now with one of the longer Ask Paul's in recent memory. Let's kick it off...

Note: We were in Oaxaca over the long weekend, so I threw in some photos from that side-trip.

? Trusting AI
wright_is asks:

Do you think we will reach a stage where we can actually trust LLMs?

Apologies for putting the end of your question first. But to answer that one, yes, most of us already do.

And I don't mean that cynically. I still struggle to use AI in my day-to-day work, which is a cute way of saying, I still don't ever use AI in my day-to-day work. But since Microsoft made GitHub Copilot free in late 2024 (with limits I've never run into), and with the rise of alternate tools like Cursor and the built-in coding capabilities in all mainstream AI models, I have started using it for my current coding project. And I discovered something interesting. Maybe two somethings.

First, the AI is so good that I just started trusting it implicitly. In the latest article in the series I'm writing about that work, I mentioned one example of how I use this almost in passing, but GitHub Copilot (in that case) basically reduced 15-ish lines of code down to a single line of code I only barely understand and could not have written myself. So that kind of success, to me, is meaningful.

Second, and tied to that, I had to test that code to make sure it worked properly. And there is a small glimpse in this of what often does not happen with AI, meaning I had to know enough about this app, and the code, to ensure that what the AI made was correct. In this case, it was. But in another similar episode, AI (I can't remember which I used, but I guess it doesn't matter) created similarly more concise code than what I had done, but in testing that code, nothing worked. Finally, I stepped through it with the debugger and discovered that the AI-created code was completely wrong. I had trusted it, stopped thinking about it, and then had to go back and fix it.

I guess there's a third finding in there, though these are getting vaguer as I go: I feel like I could easily fall into this trap almost anywhere with AI, and that it is the trust that develops over time that allows that to happen. Many people evaluated AI early on, found it lacking, and then never looked again. But many more finally went down this path, were successful, and then stopped thinking critically about what AI was doing for them. Me falling into that so easily is a bit alarming to me, as I pay attention to this stuff. Most mainstream users won't.

Anyway. Overall, my experience using AI for coding has been extremely positive. But coding is a great use case for AI because each language and framework is a finite set of data, and AI is good at grounding itself in that kind of thing. This gets more problematic in other, less concretely defined areas of life. As in...

On TWiT at the weekend, there was a discussion about AI being...