SEOMoz has posted some original research on effect of CAPTCHAs on conversion rates:

With CAPTCHA’s on, SPAM and failed conversions accounted for 7.3% of all the conversions for the 3 month period. With CAPTCHA’s off, SPAM conversions accounted for 4.1% of all the conversions for the 3 month period. That possibly means when CAPTCHA’s are on, the company could lose out on 3.2% of all their conversions!

In other words, a significant proportion of frustrated customers simply abandon their attempts to get past the CAPTCHA. (And, notably, some spam still got through!)

We’ve blogged before about the usability problems of CAPTCHA-based forms, and it’s good to see some real-world data measuring those effects.

Version 2.2.6 of the Akismet plugin for WordPress is released. WordPress users may upgrade using the Plugin Update feature, or by waiting for the forthcoming WordPress 2.8.2 release.

This release contains minor fixes only. Upgrading is recommended but not essential.

Changes since 2.2.5:

• Fix a global warning introduced in 2.2.5
• Add changelog and additional readme.txt tags
• Fix an array conversion warning in some versions of PHP
• Support a new WPCOM_API_KEY constant for easier use with WordPress MU

The download is available here.

Version 2.2.5 of the Akismet plugin is released. WordPress users can upgrade using the automatic plugin update feature.

Changes since 2.2.4:

Include a new Server Connectivity diagnostic check, to detect problems caused by firewalls

The 2.2.5 release fixes a problem in the r131686 test revision that caused spurious error reports in some circumstances.

The new Server Connectivity diagnostic feature detects problems caused by firewall or security software on some web hosts. If a web hosting company or server administrator blocks connections to Akismet.com, the Akismet plugin will be impaired or stop working altogether. The diagnostic feature will detect this and provide information that the web host can use to update their firewall to enable Akismet.

If you’re using the test revision of the WordPress Akismet plugin and seeing messages about connectivity problems, don’t be too alarmed.

Akismet is up. Our servers are working. Our network is fine. We are not experiencing any service difficulties.

We’re still sorting through the reports to find out why people are seeing error messages.

In most cases, the error messages are correctly reporting that a firewall is preventing connections from reaching Akismet.com. This problem is caused by the web host, not Akismet, and only the web host can fix it.

In some cases we’re seeing reports that may indicate spurious errors. This may be a bug in the diagnostic code, a conflict with another plugin, or an intermittent network issue at the web host – we’re still investigating.

Update: We’ve found a timing bug in the diagnostic code that can cause the Akismet plugin to report an error when in fact everything is working normally. The bug affects the diagnostic feature only; spam checking still works as normal. Version 2.2.5 of the plugin will be released shortly with a fix. In the meantime, clicking the “Check network status” button on the Akismet Configuration tab will normally remove the error message.

Update: Akismet 2.2.5 is released with a fix for the timing bug.

The issue is not a connectivity problem, but a spurious error message that is produced when connections are actually working fine.

Remember way back when you first got interested in web design? Seems like an eternity ago in web years when I made my first clumsy web sites. Maybe you got started making home pages for friends or a local club. Maybe you helped some people get hosting accounts, set up a quick web site with a forum and a blog and a shopping cart. Ah, those were the days.

It’s easy to forget that many of those old web sites are still online – abandoned, un-maintained, and insecure. Many of them date from the days before web spam was common, so they don’t include any spam protection.

Spammers know this. And they love it. Here’s why:

That’s a real screen capture from a forum that was once a real community – but has since been forgotten by its unsuspecting owner.

This is an all-too-common occurrence in recent months. In order to try to avoid being caught, spammers are hosting their advertisements for porn and pills on these old abandoned forums, which typically have an open automated registration system that lets anyone create an account without the owner’s knowledge or intervention. Then they send thousands of spam messages in blog comments and emails, with a link to those abandoned forums.

In other words: right now, someone might be sending offensive spam to thousands of blogs with a link to your web site. Like this example, taken from a comment spam (we’ve censored the real domain name for obvious reasons):

<a href="http://[REDACTED].com/forum/showthread.php?p=200789" rel="nofollow">Amateur with natural big boob</a>
-Cori the natural milf shows off her big juicy tits
<a href="http://[REDACTED].com/forum/showthread.php?p=200794" rel="nofollow">Free big tit blonde movie</a>
-Samantha babe plays with big boobs and poses in bed
<a href="http://[REDACTED].com/forum/showthread.php?p=200805" rel="nofollow">Big tit porn star movie</a>

Sadly that’s some of the less offensive spam – it took a while to find an example we could safely publish. Akismet will almost always catch these spams of course, but not everyone uses Akismet.

This technique has been around for a while, but in recent weeks we’ve seen a massive increase in the sheer number of un-maintained sites exploited in this way. Forums are the most common victims, but we’ve also seen forgotten photo galleries, blogs and social apps exploited in the same way.

Left unchecked, the damage to your reputation could be substantial – not to mention what it could do to your search engine positioning. So we’re advising everyone to please check on your old web sites.

If you have ever set up a web site for someone — or installed a test copy of a forum or web application on your own web site, even one that was never publicly announced — now would be a great time to check that it hasn’t been exploited by spammers. If it’s still clean but unneeded, consider removing it or disabling the account signup process as a preventative measure.

If you’d like to keep it online, make sure you install a spam filter like Akismet – there are Akismet plugins available for vBulletin, phpBB and most other popular web forums.

Those who like to live on the bleeding edge might like to download and test the latest revision of the Akismet WordPress plugin from Subversion:

http://plugins.svn.wordpress.org/akismet/trunk

If you don’t know what Subversion is or how to use it, I’d suggest waiting for the next official Akismet release, which won’t be far away.

The new revision includes a new diagnostic feature on the Akismet Configuration tab that’s intended to address a problem with some web hosts.

We’ve known for a while that some web hosts and servers have firewall rules that block outgoing connections — including connections to the akismet.com API servers, which are necessary for the Akismet plugin to work. Usually the host administrators will add some firewall rules to permit the Akismet plugin to connect to akismet.com. But recently we’ve discovered that some hosts have created incomplete firewall rules, with the result that some Akismet connections succeed, but some fail. This caused Akismet to seem like it was working, when in fact only some spam was checked, and only some reports ever made it back to Akismet.com.

The problem is caused by the host’s firewall rules – it’s not something that Akismet can fix. We can detect the problem, however, which is exactly what the new revision does.

The new feature adds a Server Connectivity section to the Akismet Configuration tab. The new section will check for any problems connecting to any Akismet servers, including the partial firewall problem, and recommend a course of action if there is an issue.

There’s more code in the new revision than we would typically add in an Akismet update, so testing and feedback are welcome.

Some technical details for those who are interested:

Akismet uses round-robin DNS and load balancers to distribute the work of checking comments for spam across multiple servers. A DNS lookup of rest.akismet.com (the domain used for Akismet API calls) returns not one but several IP addresses, corresponding to the multiple servers. When your blog uses Akismet to check a comment, it will (more or less) randomly choose an Akismet server IP to use. Each time it checks a new comment, it might use a different Akismet server IP.

If a web host blocks all connections to all Akismet server IPs, the Akismet plugin can easily detect the problem and report an error. The plugin has always included a connectivity check as part of its configuration process, that will warn the user if Akismet servers are unreachable.

If a web host allows connections to some Akismet server IPs, but blocks connections to the others, the connectivity check might succeed (if it happens to connect to one of the allowed IPs), while some subsequent connections will fail (when they try to connect to one of the blocked IPs). Result: the plugin appears to be working fine, but in fact only some API calls are working. Users often won’t notice anything wrong, because some spam is caught.

The new revision expands the connectivity check to include all known Akismet servers. It displays a status message or warning to the user if some servers are unreachable. It also stores a list of server IPs and their status, with a 24 hour expiry. That list of IPs is used when checking and reporting spam, to ensure that only servers that are known to be reachable are used. The list will be refreshed after 24 hours so as to adapt to any changes in availability – such as when a web host changes their firewall and blocks or unblocks an Akismet server. (Which is something that a few hosts have been known to do without warning, unfortunately).

NB: feedback about this new revision is welcome here, but please don’t post general Akismet support requests in comments. Post them here instead.

The FTC has launched legal action against a Californian web hosting service it says is responsible for botnets, malware, credit card theft and of course spam. The provider has been disconnected and its operators now face a lawsuit.

The FTC alleges that Pricewert/3FN operates as a “‘rogue’ or ‘black hat’ Internet service provider that recruits, knowingly hosts, and actively participates in the distribution of illegal, malicious, and harmful content,” including botnet control servers, child pornography and rogue antivirus products. 3FN also operates by the names APS Telecom and APX Telecom.

The provider is known as a frequent host of “scraper” or autoblog sites — fake blogs that re-publish unauthorized copies of content taken from other blogs, often sending spam pingbacks and trackbacks in the process.

Our sources indicate the network also provided services to several of the major forum and comment spammers. In particular, web sites owned by the developers of several spambot programs have been shut down (though we expect they will resurface elsewhere before long).

Of course if you’re using Akismet you won’t notice much of a difference: Akismet has long been highly effective at catching spam produced by their spambots and autoblogs.

Our stats suggest a significant and immediate drop in overall spam levels coinciding with the FTC’s action – on the order of about a 20% reduction (in spam that was or would be successfully caught by Akismet).

Akismet version 2.2.4 is now available. WordPress users can upgrade using the automatic plugin update feature.

Changed in this version:

• Fixed a key problem affecting the stats feature in WordPress MU
• Provide additional blog information in Akismet API calls

The extra blog information passed to the Akismet API will help Akismet to better adapt and provide results that are tailored to your specific blog.

Support: Please use the Akismet support form to ask questions or report possible problems. Support questions posted in comments will be removed.

Update: the problem has now been fixed.

A bug in development versions of WordPress is causing some comments to be incorrectly caught as spam. The problem is in WordPress, not Akismet, and there’s no way for Akismet to prevent it from happening.

The problem only affects WordPress blogs running current development code. It does not affect other applications that use Akismet.

Technically-minded users can read the specifics in the WordPress Trac system.

We expect to have a fix in WordPress core soon, and deployed to WordPress.com shortly after.

This is probably a good opportunity to remind readers that Akismet is not necessarily the reason a comment gets caught as spam. The majority of complaints we receive about real comments being caught as spam were in fact not caused by Akismet at all – but by other spam filtering plugins or features. (Unfortunately WordPress doesn’t show the reason a comment was moved to the Spam filter, so there’s no easy way to tell which ones were put there by Akismet and which ones by something else).

In particular, the WordPress Comment Blacklist feature (Settings / Discussion) trips up some users. It lets administrators provide a list of words, IP or email addresses that should be blocked as spam. Any comments that match words in the blacklist will automatically be moved to the Spam filter – regardless of whether or not Akismet considers it to be spam. The blacklist matches within words, meaning that if you add a short string such as “ru”, it will block any comment containing the words “truth” or “fruit”, or any other word with the letters “ru”. And, since the blacklist takes precedence over Akismet, reporting those comments to Akismet as false positives won’t stop them from being caught.

If you think the Comment Blacklist feature might be catching legitimate comments as spam, the tw-blacklight plugin might help your diagnosis.

We’ll update the Akismet blog when the WordPress bug has been fixed.

One of the most common forms of comment and pingback spam right now is the relatively subtle, ambiguous kind — short phrases or questions that are not obviously spam, at least on face value. Since we last posted about this, the more sophisticated spammers have progressed from old standbys like “nice post” and “great blog”, to more cunning things like questions (“where can I download your theme?”) and appeals to your helpful nature (“I’m having trouble subscribing to your RSS feed”).

Akismet almost always catches these kinds of bogus comments.

The tip-off of course is that they often include a link to a site that’s advertising dubious or sleazy merchandise (or worse, a web site that harms the viewer’s computer). But it’s easy to forget to look at the link before approving a comment, or give the comment author the benefit of the doubt without checking closely. And spammers have recently learned to post several comments over time, the first of which contains no link or obvious clue. (We call these precursor spams).

Anyway, a comment is a comment, right, so what’s the harm in approving a few tame platitudes, even if they were posted by spammers?

Unfortunately it is harmful, and most of the damage is to your own site.

By moving these comments out of your spam folder and publishing them on your blog, you’re doing three things, all of them bad:

1. You are undermining your site’s SEO.

The spammer’s web site might seem inoffensive on face value. But the black-hat SEO and spam methods used by its promoter are not. That same spammer is busy building backlinks from anywhere they can find them, including some of the web’s worst neighbourhoods. By regularly publishing links to spammers’ web sites, you’re giving Google and other search engines a hint that links from your blog are poor quality.

Now it’s true that Google will try not to penalize a web site for inadvertently linking to a bad neighbourhood. But even if they don’t, you are weakening the value of each of the other links from your blog – “diluting your GoogleJuice”, if you like – and helping to validate the spammer’s web site. In some cases you might even find that you are helping the spammer overtake your blog in search engine results.

2. You are attracting more spammers.

Less skilled spammers will deliberately seek out blogs that other spammers have successfully spammed, because they know they are easy targets. Organized spammers circulate lists of such blogs (for a small fee of course). And professionals keep their own lists of previous victims, because they know future spam is even more likely to be approved there. By letting some spam through – even seemingly harmless ones – you are providing a signal to spammers that your blog is a profitable target. (Experienced bloggers will be familiar with this phenomenon: you accidentally approve one seemingly unremarkable spam comment, and a big batch of ugly spam follows soon after).

WordPress and many other blog applications have a feature, independent of Akismet, where regular users who have had at least one comment approved, will automatically skip the moderation queue next time and have their comments published right away. Spammers know this, and they’ll come back to take advantage of it. Often they’ll link to a harmless looking site in their first comment (or include no link at all), but link to progressively more blatant spam in subsequent comments.

3. You are damaging your reputation.

You might not click on the links in all the comments on your blog, but some of your readers will. And some of those links will go to sites that are sleazy, offensive, or harmful.

Worse still, a spam tactic that is becoming more popular is to first post a small number of spam comments on innocent blogs; then send a large volume of spam to other web sites linking to the blog post that contains those comments. (They do this to try to get around spam filters and blacklist that recognize and catch links to their own site).

If you do publish spam comments on your blog, you might discover later that thousands of other blogs and forums have been spammed with links to your blog.

So what should you do about it?

Akismet will almost always catch these comments and put them in your Spam folder. Usually you don’t need to do anything; just don’t approve them for publication.

We have a real-time view of spam activity on millions of blogs around the world, so we can detect patterns in behaviour that can’t be seen by looking at any one single comment. If a bland, generic comment turns up in your spam folder, you should be suspicious of it – Akismet flagged it for a reason. Think twice before approving it for publication. Unless you know the author, it almost certainly is spam — or a subtle precursor to it.

Also, keep an eye out for forthcoming Akismet updates. In addition to our usual work behind the scenes monitoring and adapting to new spam techniques, we’re developing some new features designed specifically to help protect against the potential harm done by spammers.