by Richard DeMillo

If you can read the text strings below, chances are pretty good you’re human.

These five graphics were plucked a few days ago from the sign-up pages of the most visited sites on the Internet: Google, Yahoo, MySpace, YouTube, and FaceBook. These sites, and virtually every other high traffic site on the Web, use “Captcha” (Completely Automated Public Turing test to tell Computers and Humans Apart) as the first line of defense against the onslaught of Virtual Blight from black hat marketers and distributors of malware.

The problem? Captchas don’t work anymore. Over the past six months, programs have been designed to automatically solve Captchas. This is exactly what the tests are designed to avoid: automatic solutions. A group of programmers in Russia announced a technology that could solve some of the most difficult Captchas 35% of the time. Another system, EZ-Gimply, developed at UC Berkeley, claims to solve a tough class of Captchas 92% of the time. Recently, it was reported that Spammers have cracked Microsoft Live’s Email account creation.

Why do Captchas matter? Looking at data streams that do not use Captcha, such as email and blog comments, signal to noise ratio is unbearable; 9 out of 10 bytes of email traffic and 91% of comments on blogs are Spam. By blocking the automated creation of accounts and postings, Captcha’s throttle back the volume of blight by an order of magnitude. Instead of grappling with a 9:1 ratio of spam to legitimate traffic (let’s call it “Ham”), websites that gate participation by using Captchas typically see only 1 byte of Spam for every 9 bytes of Ham. This is a manageable level, small enough to be further reduced by more labor-intensive strategies, like community flagging systems, volunteer editors and special-purpose redaction bots.

“Captcha plays an essential role as the first line of defense for Web 2.0 sites against vandals and marketers trying to exploit user generate content sites, blogs and opinion sites to hawk their products. Sophisticated spammers have already developed reliable ways to get around Captcha for about $.01 each and it only a matter of time before these techniques trickle down and Captcha becomes less of a nuisance to hackers than it is to legitimate users. Unless we develop a new solution, Social Media, Social Networking and Community sites will be as filled with blight as your inbox without a Spam filter.”

Captchas are like a leaking levee facing a hundred-year storm. As mentioned, Bots with sophisticated character recognition and parsing components are able to solve over 35% of them. Because bots are so scalable, that’s a frightening success rate. It means that three times the volume of attacks would essentially erase the protection that Captchas currently provide. Worse, the roadmap for improving Captchas is bleak. Internet security in general is a cat and mouse game, with security systems and professionals trying to stay a step ahead of the black hats. But staying ahead with Captchas is tricky and may be impossible.

No Admittance. To work, Captchas, need to do more than foil the bots; they also need to let in the people. Look again at the samples at the top of this article. The second and fourth ones, “V3YG” and “swable,” are pretty easy to read. The other three are much harder. In making Captchas more difficult, site owners create “false negatives” – instances where a real human being fails to read the Captcha correctly. Currently, major websites see a false negative rate of about 20%. In other words, two out of ten captchas foil real users, while fake users can foil the Captchas more than three out of ten times.

The levee is starting to fail.

The greatest threat from Virtual Blight is to Web 2.0, the community web. YouTubes, Wikipedias, MySpace, Facebooks, Digg, etcetera, are based on users creating the content. The value of Web 2.0 sites is based on trust. Users trust that other users are legitimate community members. Posts are from people with opinions, polls reveal what people think, friending requests are from people who want to be your online friend.

What’s needed is an alternative type of Automated Turing Test. A test that is less intrusive and more reliable. A new line of defense that guarantees a human presence at the other end of the click, be it on a submit button, a link or banner, a seat at a poker table or a profile in a social network. Without a new approach, nine-tenths of the signal could soon be noise and trust and usability will be washed away in a flood of automated marketing blight from sock puppet accounts.

Richard DeMillo is Dean of Computer Science at Georgia Tech and former Chief Technology Officer for Hewlett-Packard. Prior to HP, DeMillo has also directed the Computer and Computation Research division of the National Science Foundation.

This entry was posted on Tuesday, February 12th, 2008 at 1:09 pm and is filed under Articles. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.