New plugin version 2.2.4

Akismet version 2.2.4 is now available. WordPress users can upgrade using the automatic plugin update feature.

Changed in this version:

  • Fixed a key problem affecting the stats feature in WordPress MU
  • Provide additional blog information in Akismet API calls

The extra blog information passed to the Akismet API will help Akismet to better adapt and provide results that are tailored to your specific blog.

Support: Please use the Akismet support form to ask questions or report possible problems. Support questions posted in comments will be removed.

WordPress bug

Update: the problem has now been fixed.

A bug in development versions of WordPress is causing some comments to be incorrectly caught as spam. The problem is in WordPress, not Akismet, and there’s no way for Akismet to prevent it from happening.

The problem only affects WordPress blogs running current development code. It does not affect other applications that use Akismet.

Technically-minded users can read the specifics in the WordPress Trac system.

We expect to have a fix in WordPress core soon, and deployed to shortly after.

This is probably a good opportunity to remind readers that Akismet is not necessarily the reason a comment gets caught as spam. The majority of complaints we receive about real comments being caught as spam were in fact not caused by Akismet at all – but by other spam filtering plugins or features. (Unfortunately WordPress doesn’t show the reason a comment was moved to the Spam filter, so there’s no easy way to tell which ones were put there by Akismet and which ones by something else).

In particular, the WordPress Comment Blacklist feature (Settings / Discussion) trips up some users. It lets administrators provide a list of words, IP or email addresses that should be blocked as spam. Any comments that match words in the blacklist will automatically be moved to the Spam filter – regardless of whether or not Akismet considers it to be spam. The blacklist matches within words, meaning that if you add a short string such as “ru”, it will block any comment containing the words “truth” or “fruit”, or any other word with the letters “ru”. And, since the blacklist takes precedence over Akismet, reporting those comments to Akismet as false positives won’t stop them from being caught.

If you think the Comment Blacklist feature might be catching legitimate comments as spam, the tw-blacklight plugin might help your diagnosis.

We’ll update the Akismet blog when the WordPress bug has been fixed.

Eliminating spam is good SEO

One of the most common forms of comment and pingback spam right now is the relatively subtle, ambiguous kind — short phrases or questions that are not obviously spam, at least on face value. Since we last posted about this, the more sophisticated spammers have progressed from old standbys like “nice post” and “great blog”, to more cunning things like questions (“where can I download your theme?”) and appeals to your helpful nature (“I’m having trouble subscribing to your RSS feed”).

Akismet almost always catches these kinds of bogus comments.

The tip-off of course is that they often include a link to a site that’s advertising dubious or sleazy merchandise (or worse, a web site that harms the viewer’s computer). But it’s easy to forget to look at the link before approving a comment, or give the comment author the benefit of the doubt without checking closely. And spammers have recently learned to post several comments over time, the first of which contains no link or obvious clue. (We call these precursor spams).

Anyway, a comment is a comment, right, so what’s the harm in approving a few tame platitudes, even if they were posted by spammers?

Unfortunately it is harmful, and most of the damage is to your own site.

By moving these comments out of your spam folder and publishing them on your blog, you’re doing three things, all of them bad:

1. You are undermining your site’s SEO.

The spammer’s web site might seem inoffensive on face value. But the black-hat SEO and spam methods used by its promoter are not. That same spammer is busy building backlinks from anywhere they can find them, including some of the web’s worst neighbourhoods. By regularly publishing links to spammers’ web sites, you’re giving Google and other search engines a hint that links from your blog are poor quality.

Now it’s true that Google will try not to penalize a web site for inadvertently linking to a bad neighbourhood. But even if they don’t, you are weakening the value of each of the other links from your blog – “diluting your GoogleJuice”, if you like – and helping to validate the spammer’s web site. In some cases you might even find that you are helping the spammer overtake your blog in search engine results.

2. You are attracting more spammers.

Less skilled spammers will deliberately seek out blogs that other spammers have successfully spammed, because they know they are easy targets. Organized spammers circulate lists of such blogs (for a small fee of course). And professionals keep their own lists of previous victims, because they know future spam is even more likely to be approved there. By letting some spam through – even seemingly harmless ones – you are providing a signal to spammers that your blog is a profitable target. (Experienced bloggers will be familiar with this phenomenon: you accidentally approve one seemingly unremarkable spam comment, and a big batch of ugly spam follows soon after).

WordPress and many other blog applications have a feature, independent of Akismet, where regular users who have had at least one comment approved, will automatically skip the moderation queue next time and have their comments published right away. Spammers know this, and they’ll come back to take advantage of it. Often they’ll link to a harmless looking site in their first comment (or include no link at all), but link to progressively more blatant spam in subsequent comments.

3. You are damaging your reputation.

You might not click on the links in all the comments on your blog, but some of your readers will. And some of those links will go to sites that are sleazy, offensive, or harmful.

Worse still, a spam tactic that is becoming more popular is to first post a small number of spam comments on innocent blogs; then send a large volume of spam to other web sites linking to the blog post that contains those comments. (They do this to try to get around spam filters and blacklist that recognize and catch links to their own site).

If you do publish spam comments on your blog, you might discover later that thousands of other blogs and forums have been spammed with links to your blog.

So what should you do about it?

Akismet will almost always catch these comments and put them in your Spam folder. Usually you don’t need to do anything; just don’t approve them for publication.

We have a real-time view of spam activity on millions of blogs around the world, so we can detect patterns in behaviour that can’t be seen by looking at any one single comment. If a bland, generic comment turns up in your spam folder, you should be suspicious of it – Akismet flagged it for a reason. Think twice before approving it for publication. Unless you know the author, it almost certainly is spam — or a subtle precursor to it.

Also, keep an eye out for forthcoming Akismet updates. In addition to our usual work behind the scenes monitoring and adapting to new spam techniques, we’re developing some new features designed specifically to help protect against the potential harm done by spammers.

Make Commenting Easy

Back in the early days of blogging and when comment spamming was still fairly small scale (compared to today) one method people used to stop a spambot was to use a CAPTCHA. The idea was that a comment spambot could not read the image and so the comment would fail and you would not get any spam. Obviously the spammers really did not like these CAPTCHA’s so they devoted resources to get around them.

In January 2004 — 5 years ago — Cory Doctorow blogged about pornography being used to get humans to solve captchas for spammers and there were scripts which could defeat different CAPTCHA’s. It is because of those scripts that you can now be faced with images that you struggle to get right. All you want to is leave a comment right?

Companies still believe in the power of the CAPTCHA and they are now very wrong to do so. Where there is a demand  by those wanting to spam there is supply – and it’s less than a cent to spam your blog. At ZDNet’s Security blog they report on an industry which can solve a quarter of a million CAPTCHA’s a day.

You write a post and you would like comments. Using a CAPTCHA to stop a spammer is not going to work. If someone is writing a reply to your post why make them solve some image with distorted letters? They want to think about your post not whether that is an 8 or a B, a 1 or I or l. If people have previously been annoyed by these things they may not even bother trying to leave a comment. You lose here — your blog lost a comment.

Putting visible obstacles to commenting irritates readers and gives spammers something to overcome and the more information spammers have the more likely they are to spam your blog. This is why Akismet works as it does — keep the spammers guessing but let people comment freely. Commenting should be as easy as blogging because that is how to keep your conversations going.

Stats accuracy improved

We’ve improved the accuracy of the Akismet stats feature that was introduced in Akismet 2.2.1. You might notice a slight change in the total spam, comments and accuracy figures reported at the top of your stats page, and the percentages shown in the pie chart. The Historical Stats figures and time graphs are unaffected – these were correct all along.

There’s no plugin upgrade required.

Our overall spam statistics remain unchanged, and Akismet is as accurate as ever.

Congrats Defensio

Just a bit after a year after we welcomed Defensio to the market it’s just been announced that that they’re being swallowed by the public security company Websense.

Given the size of Defensio’s team and user base it probably wasn’t a huge acquisition, but this is still an important validation of the  space in general as traditional email anti-spam and security companies have been completely blind to web spam thus far, and in my opinion it’s the fastest growing threat to most businesses today.

(Think about it: if you get a spam in your email, it’s annoying but not the end of the world; if you get a spam on your website it’s immediately visible to your customers who probably won’t be impressed by “buy viagra’s” contribution to the conversation when thinking about your products or services.)

2009 is shaping up to be a pretty interesting year in our world. We’ll definitely be keeping an eye on the ever-increasing players in this market, but more importantly we’ll be listening to you guys and do our best to be as invisible and effective as possible. (Especially as we approach 10 billion spams blocked!)