Ask Paul: November 18 (Premium)

Happy Friday! Let’s get the weekend off to an early start with another blockbuster round of reader questions.
Our future AI overlords
wright_is asks:

With the increasing use of AI to recognise NCMEC [National Center for Missing & Exploited Children] and the increasing reliance on online services, there are some serious problems that are coming at us.

There have been cases of people having their accounts locked, because they took pictures of their children to send to their doctor and their phone automatically uploaded a copy to Google Photos.

In Germany, someone was out with his sister and her family at the beach. She borrowed his phone, because it had a better camera, to take pictures of her children playing on the beach, being small children, they weren’t wearing bathing suits.

His phone uploaded the images automatically to OneDrive. The next morning, he found that his account had been locked. No warning, no information about why it was locked. He filled in the form on the Microsoft website, but the reply was that he had broken the terms and his account was locked, end of story, no information on what rule he had broken.

No email, no Xbox Game Pass, no access to purchased games, no access 10 years worth of files on his OneDrive, no access to Office. Nothing, everything that he had paid for or was paying for, gone. No explanation, no way to get it back.

Finally, after 18 months, he found out what had happened, when he received a letter from the police, that he was being investigated for distributing NCMEC. He went for the interview and was confronted with the images his sister had taken of her children playing on a public beach. The police dropped all charges, but that wasn’t enough for Microsoft, his account remains locked, with no way to get it re-opened, after their AI miss-classified family photos and reported him to the police.

What can somebody do in such a situation?

Right now, all we can do is call attention to these issues and hope that Big Tech understands that this is one of many areas where human oversight is and probably always will be necessary in this push toward the new automation, which is AI. And that they are not the overlords of the law: that is the responsibility of government and law enforcement.

A couple of thoughts.

You’re right in that this happens all the time: I just saw a fascinating and disturbing story (I think in my Pocket feed) about an astronomer who tweeted a video of a meteor shower and was subsequently banned from Twitter for posting "intimate content." The ban was initially for 12 hours but then it just continued and she exhausted her appeal options over 3 months. And this was before Elon Musk destroyed Twitter and fired all the moderators. Ironically, after the BBC published an article about this incident, her account was finally restored. But many other astronomers and others posted similar photos and videos of the same event---some, the exact same video---a...