Behind Thurrott.com: Comment Moderation

As you probably know, we use OpenWeb for content moderation on Thurrott.com, and I couldn’t be happier with the service. But I met recently with OpenWeb, and now I’m even more jazzed about the service. More to the point, I will be able to better address some common feedback.

I don’t recall when we switched to OpenWeb—it was over a year ago—but what I do remember, quite clearly, is the sense of calm that descended over me when it happened. Previous to that, we used a homespun commenting system that a previous web developer had created for us. And as was the case with all of the commenting systems I’ve dealt with over the course of my career, it did nothing to prevent the spread of spam, leading to an hour or more of tedious management work each morning.

Of course, you approach this site as a reader, and so you may have never considered what the day-to-day non-writing work is like for Laurent and me. At a high level, you probably understand that we must read the comments that our articles generate. After all, we often respond to them. But there’s more to it than that, and our use of OpenWeb has really changed things in a good way. There’s a much more sophisticated system now.

Part of it is emailed-based: when someone likes or responds to a comment I’ve made, OpenWeb emails me. This is super-useful because I can deal with it no matter where I am, including out in the world on my phone, by following up with a “like” or a reply.

But most of it happens on the site, of course. One thing I can do, and had done for years, is simply scan the latest articles on the home page, see which of my articles have comments, and then open each in a new tab to read them and follow up if needed. But this approach has two obvious flaws: I need to look at all comments, not just the ones to my articles, and people will often comment on articles that are more than a day old. And so I don’t typically do that anymore.

Instead, I use the OpenWeb management console. There are two main views, and I check them both repeatedly throughout the day: Pending and Approved.

The latter is pretty straightforward, but Pending is a bit more nuanced than it may initially seem. Pending comments are one of two things: comments that I need to approve before they go live on the site, which probably makes sense to everyone, but also some rules-based list of comments that have actually gone live but I can still review.

The best part about Pending, to me, is that it catches spam. People often create an account specifically so that they can post spam links in a comment, never to return. But we don’t allow first-timers to post comments, so they end up in Pending, and that means that all the (comment) spam gets caught. (Actually, all the forum spam, where a new account creates a spam post, gets caught too, and for the same reason, though this is done through WordPress, not OpenWeb).

It’s hard to express how big of a deal this is to me. With rare exceptions, a spammer will post only a single spam comment or forum post now because they never go live. In the past, they would see that the first one was successful and then start spamming the site. I used to wake up to mornings in which I’d waste a lot of time, sometimes over an hour, clearly through that crap. Now, there are only a few each day on average. Some mornings, I wake up to no spam. Bliss.

(We also configured OpenWeb to not allow guest comments, likes, or reports. Guests are people without an account, free or Premium. Same rationale.)

What I just described is probably possible using stock WordPress. But OpenWeb does more. So much more.

The one feature that I know many readers are familiar with, because they’ve run afoul of it, is OpenWeb’s AI-based moderation system: this system detects toxic comments automatically, and it can be configured to handle comments with images and external links in whatever fashion. In the beginning, we went with the default configuration, figuring if OpenWeb was good enough to handle the toxicity at Fox News, it could surely handle our site. But what we heard, again and again, was agitation with its rejection of comments with adult language.

I asked whether there was some dial we could spin on that, but was told no. And so, looking through the management console, I came across a Restricted Words interface that literally lists all of the words and variations that it blocks. And so as users complained, or, more frequently, as I saw rejected comments in the queue, I would see which words triggered the block and remove them from the list. I honestly figured that this would eventually work in the sense that there are words like crap that should be fine in adult conversation, and that I’d quickly eliminate the least offensive of the restricted words.

But in a meeting with OpenWeb, I found out that there is indeed a dial—actually, many dials—that I can use to adjust the sensitivity of the moderation across a wide variety of filter types. And I have been experimenting with these dials, judiciously, since I started running things here.

There are filters I want to set on “high” (strict), like Civility, Attack on Commenter, Insult, Toxicity, Severe Toxicty, and the like. But there are also filters I can set to “low” (less strict) or “medium.” I moved Profanity and Obscene to “medium,” for example.

And since this came up recently, I can tell you that comments with images and comments with external links are both set to “Require Approval.” One option would be to configure one or both to “Publish & Moderate,” which would let them go live immediately but also display them in that Pending list so I could review them each day. (Comments, generally, are set to “approval all,” so those are two exceptions.) Let me know what you think about that.

I’m asked about shadow banning from time to time. I had never heard this term until sometime in the past 6 or 12 months, I don’t recall. But OpenWeb does have a shadow ban feature that’s hilariously called “Bozo mode,” in which the impacted individuals will see their comments on the site, but no one else will. We do not use this feature and I am opposed to it. Instead, we use something called “Clarity mode” that aligns with my desire for transparency: in this mode, commenters can see the status of their comments in conversation and are notified when any comment status has changed.

I just noticed that we have two sets of Community Guidelines, which explain what is and is not allowed. There’s one version on the site and then one in OpenWeb that I think is better but can’t figure out where it’s ever displayed, if ever. So I will engage with the web dev team to merge those.

Right now, we have several Thurrott.com and BWW employees listed as “teammates” in the OpenWeb admin console, and right now all of them are admins, which I’ll be changing soon. I feel like Laurent, George, and I (and the web devs) are the only people who need to be admins, so I’ll make the rest moderators soon. But this system also affords us the possibility of adding readers to a role called Community Moderator which would let them exert some control over comments from the article pages (but not access the admin console). A few people have expressed interest in doing so over the past few years, and I’m not against it. But I also want to balance this with my need to read all the comments. This one is overdue, but I’ll figure out something soon.

Beyond the functionality that we’re using today, OpenWeb offers other features we can add to the site, and some of them are of interest.

In a recent Ask Paul, for example, someone asked about the possibility of a Thurrott.com Discord channel, and I noted in my answer that Twitter’s stupid upheavals are now preventing us from displaying my Twitter content on the site, and that this is a problem for a number of reasons, one being that I live-tweet Apple, Google, and Microsoft events. And so I don’t see us going the Discord route, exactly, but OpenWeb does offer message broadcasts (a way to send an instant notification to all users on the site, along with a link) and embeddable (and persistent) live blog capabilities, which means I should be able to do that off-the-cuff live coverage right here on the site instead of on a third-party service that’s being run into the ground in real-time. So we’ll be experimenting with that soon.

And so that’s most of that, I think. I have other articles planned for this series—the next one is about how we actually deal with spammers—and I will be as transparent as possible in the running of the site, just as I am in the rest of my life. And now that the transition is pretty much complete, with the new newsletter, I am going to go back to our normal schedule of monthly site updates, accompanied by blog posts like this one, too. We’ve had a lot of feedback in the forum and via [email protected] in recent weeks about changes and fixes that readers would like to see, and Laurent and I will go over those soon (and then regularly), identify the most important ones to implement first, and do our best to make this site as good as it can be for you. That’s what it’s all about.

So thanks for reading, and keep the (constructive) feedback coming. This site is in a good place right now. But I know it can be even better.

Paul