AI is like…

I was listening to the re-branded This Week in Google, now Intelligent Machines, last week, with their special guest Stephen Wolfram and he made a very good comparison for AI.

When you build a wall, you can use two methods, 1) you use regular shaped bricks and can build a high and solid structure with it or 2) you can use stones laying around and press them together and find stones that fit gaps to make a low wall that is somewhat sturdy (thing of English drystone walls, such as you find in the Yorkshire Dales). The latter are fine for a wall around a field, but they need regular maintenance and can’t be used to build a high, stable wall or structure.

AI is like the latter, it looks around for “stones” that are laying around and tries to put them together. This works to a certain extent, but it is no guarantee that you will get a coherent answer.

Listening to his talk, it suddenly dawned on me, why I have such a problem with AI. I come from a scientific and engineering background, I expect concise and provably correct answers to questions. Even when I was a programmer in the past, the code produced provable results and, if there were bugs, you could easily identify them and correct them, so that the program gave the correct results.

AI just doesn’t work like that, it grabs random stones, sees that, according to the rules it has learnt, that they should fit together and shows them as the answer. Language has rules, noun, verb, noun verb, with the odd adverb and adjective thrown in for good measure, but a human can take those basic rules and work out when those constructs are producing garbage. An AI can’t (yet) and can’t tell whether what it is saying makes sense, let alone what it is saying is true or not. This sort of lackadaisical way of producing results just goes against the grain for someone coming from a domain that requires concise and accurate answers.

I like AI, I think it is interesting, but every time I use it, I start to get enthusiastic about it, then after a few answers, it proves to me that it is completely useless by producing error ridden answers to questions I wanted confirmation of. If it can’t reliably answer questions on subjects where I just need confirmation, how am I supposed to trust it to provide answers to questions on new, to me, subjects?

2 weeks back, I was making a list of the PCs we need to replace over the next 4 years and I started using AI to answer the question, “which generation of Intel Core processor does a PC-maker PC-model use?” The first 5 or 6 answers were sensible and close enough to what I was expecting that I took them at face value, then I asked the same question for the Dell Optiplex models we received last year (I wanted to know if they were 11th, 12th or 13th generation processors), the AI answered “The Dell Optiplex 7010 uses 2nd generation Core processors”! Erm, no, I’m 100% sure that Dell doesn’t currently sell any new PCs with a 2nd generation Core processor! Going to the Dell website and entering the Tag number from one of the PCs confirmed it was using a 13th generation Core processor.

But such a bad answer caused me to waste an hour going back over the other questions and double checking them against the manufacturers website or reviews using a normal search engine (some of the machines were so old they were no longer listed on the manufacturers website and finding a review of the correct model wasn’t always easy, either), just to make sure that those answers were actually correct.