A Tale of Three AIs (Premium)

We’re barely one year into the generative AI era, but we’re finally starting to understand the ways in which this technology can improve our productivity. Part of the hurdle here is our natural tendency to make AI work in familiar ways. But AI is well-suited to certain things and less well-suited to others. And trying to make it work like the tools of the past is a mistake.

With the understanding that my findings aren’t particularly unique, I do have some general observations about how generative AI might be best used. These observations are limited in some ways to my needs—for example, I feel that Copilot, DALL-E, and Gemini are particularly good at creating images from text prompts—and will evolve as the technology evolves and my experience grows.

Generative AI is particularly good for summarizing. This could be a lengthy email, a post-meeting wrap-up, a long article on the web, a PDF, whatever. This isn’t about laziness, though it will be used by the lazy. It’s about making the best use of your time, finding what’s pertinent or the most important, and focusing on that. The implementations vary, but I like the side-by-side approached used by Copilot for Microsoft 365, and how it integrates with core Office apps like Word and Outlook.

Generative AI works best with a limited data set. This is the grounding feature I discussed recently. The more data that AI has, the more inaccurate it will be, and the more it will “hallucinate” and present untruths (or even purely invented information) as fact. Products like Copilot for Microsoft will use grounding to limit AI’s reach to an organization, but individuals will see this in things like custom GPTs that are either provided to us by AI makers or third parties, or things we make ourselves. They will only—and can only–get better in time.

Finally, generative AI works better when you’re more specific. This is the key conceptual hurdle, I think, and the key learning: We’re trained to use traditional search engines like Google’s to get answers to specific questions (“What is the capital of Massachusetts?”), but that’s not an ideal use-case for AI. (Even though almost any AI would handle that particular question; “What happened yesterday in Mexico City?” would be more problematic.)

We’re so trained to be terse in our interactions with these things that it probably doesn’t even occur to most people to behave otherwise when dealing with AI. This is true of image creation, which doesn’t impact too many people. But it’s also true of any generative AI capability. Including those that would benefit a broader range of users. Including the productivity scenarios that really matter.

The earliest theoretical example I used to demonstrate, if vaguely, was how one might use it to perform an unfamiliar task like creating a PowerPoint presentation. In this scenario, I imagined a typical knowledge worker who perhaps spent their days writing text in Word or crunching numbers in Excel. And then one day, the boss sticks his head in his office doorway and tells this person that they need to make a presentation at some coming meeting. And this person is not familiar with PowerPoint.

This example didn’t require a lot of imagination, it’s something that happened to me many times, though the boss and office doorway are apocryphal in my case. I happen to be a writer. And through no efforts on my part, I have sometimes had to give presentations. I do not like giving presentations. I am not familiar with PowerPoint. I have always dreaded these times.

But what if AI could help? Rather than do what I was forced to do in the past—buy a book, perhaps, research the product online, watch videos, or whatever—what if I could simply direct AI to create this presentation for me? Not the whole thing. But rather the skeleton of a presentation, with a nice look and feel, something I could edit and personalize with my own content. I am writer, so I can do that much. But why waste time mastering a tool like PowerPoint that I will use only once or every once in a while?

(Less apocryphally, I create a PC sales chart in Excel every January to accompany my annual story about the PC sales in the previous year. This is almost always the only time I open Excel each year, and each time I do this, I have to relearn how to update the chart. It’s not only not a skill, it’s just something I don’t care about.)

I was discussing this use case with my wife the other day and how generative AI works so much better when you’re specific. And how I was testing three different paid AIs—Microsoft Copilot Pro, OpenAI ChatGPT Plus, and Gemini for Google Workspace—and that each was, in different ways, better or worse than the other. These differences are a moving target, since each product evolves on what is probably a daily basis. But I thought it might be fun to make a specific comparison that combined this testing with my understanding of how much better each works when you’re specific. And so I invented this admittedly random test to see how each would fare.

That is, I would ask each of these things to make a presentation. Or, at least the bones of a presentation. I would approach this not as I might personally do so, but rather in a more common scenario, like a student with an assignment. And so this presentation would not be something I would have to present. Just an example.

And what I came up with was the following prompt.

‌I need to create a presentation with a title slide, 10 content slides, and a thank you slide with contact information at the end. Each of the 10 content slides should include a famous quote from a famous individual, plus a representative photo and/or background image. The famous people should include Steve Jobs, Bill Gates, Nelson Mandela, Oprah Winfrey, Benjamin Franklin, Franlkin D. Roosevelt, Martin Luther King, Anne Frank, and Ronald Reagan.

The results were fascinating, a slice in time look at where each of these services is today.

I tried Gemini first, which was perhaps unfair: This text-based chatbot works like Copilot or ChatGPT, so I knew—or at least assumed—that it wouldn’t push me over to Google Slides and would instead just create a text-based representation of this presentation. And that’s exactly what it did.

The organization here is nice, with slide titles and headings on each content slide for title, quote, image, and even speakers notes. And some of those notes are great: “Discuss Jobs’s reputation as a visionary entrepreneur, driven by a relentless pursuit of innovative technology. Explain how the quote highlights the necessity to break new ground rather than merely imitate existing models.” Nice.

ChatGPT provided a similar response, but with a little less detail. Most importantly, there are no speaker notes, though this is tied to my “be specific” rule, above. I didn’t ask for speaker notes, and I could have. (And should have. That’s something to add after the initial results betray a hole in the initial request.)

With Copilot Pro, I can actually use Copilot in PowerPoint to create a presentation, and I was excited to see what it came up with. Sure enough, it came up with a reasonably professional presentation. At first glance, this looked great.

But Copilot didn’t follow my instructions: I had asked for a quote from each person to be the centerpiece of each slide, but the content slides it created read more like speaker notes. More to the point, they don’t even include a famous quote.

This reminded me that Google Slides might be able to do the same thing. And so I turned to Google’s presentation software and prompted that too. But that didn’t go well at all: Gemini, at least for now, refuses to create anything related to people. Which seems ludicrous to me.

A couple of quick points here.

None of these AIs provided the photos I asked for. Most of them at least provided the quotes. And I give Gemini bonus points for adding nice speaker notes, and then dinged points for refusing to actually create the presentation in Google Slides. But each of these things did some good work with the prompt I provided, and each would be a good starting point for further customization and specificity.

But lost in all this is a harsh reality: I didn’t do this work, it’s a made-up example, but I would need to ensure that those quotes are all real, and accurate. It’s possible some are made up, were said by other people, or are inaccurate in other ways. That’s true of any and all the content these things created.

And each service will work differently, and hopefully better, tomorrow. And next week. And later this year. And so on.

That’s the nature of AI. It’s a moving target. And to me, not just utterly fascinating but endlessly useful. This is obviously just the tip of the proverbial iceberg, the start of an exciting new age.