Cleverbot comes very close to passing the Turing Test

Techniche 2011, IIT Guwahati, India, 3rd September 2011

A high-powered version of Cleverbot took part alongside humans in a formal Turing Test at the Techniche 2011 festival. The results from 1,334 votes were astonishing...

Cleverbot was judged to be 59.3% human.

The humans in the event achieved just 63.3%.

"It's higher than even I was expecting, or even hoping for. The figures exceeded 50%, and you could say that's a pass. But 59% is not quite 63%, so there is still a difference between human and machine." Rollo Carpenter

During the event people voted how human-like responses seemed, from 0 to 10. Thirty of the audience volunteered, and chatted on 3 screens in 10 rounds of 4 minutes each. Half the conversations were human-human.

The Turing Test was proposed by early computer scientist Alan Turing. 'Thinking' and 'intelligence' are hard to define, so he suggested a test, with people and machines communicating via text. If a person could not tell human from machine more than half the time, the machine should be called intelligent.

Turing made a prediction that in 5-minute conversations machines would pass the test 30% of the time by the year 2000. That mark has now surely been passed. And with 30 conversations, 100 separate voting individuals and a total of 1,334 votes cast, the Turing Test at IIT Guwahati holds mathematical significance. Even so, different tests with different people and circumstances can and certainly will see different results.

"It was mesmerising to watch a bot chatting just like a human, and people finding it so hard to distinguish." One of the humans at the event

Our AI has been learning online for 15 years, but has recently seen an exponential growth curve of data and visitors. It is normal for there to be 100,000 conversations a day, each with 20 or more interactions. There will have been a billion things said to Cleverbot by the end of this year alone.

Cleverbot was given more processing power for this test than it can be online. It had two dedicated, fast computers with solid state drives while talking to just 1 or 2 people at once. Online there are often 1000 people talking to each machine. We know you'd all love to talk to it the powerful version, but we need a lot more servers first!

THANK YOU to all Cleverbot users for helping it to learn. We intend to fully, unquestionably, pass the Turing Test, with new resources and extensions to the AI on the way.

Some press coverage of the event

Software tricks people into thinking it is human   New Scientist, 6th September 2011

How the Cleverbot computer chats like a human   MSNBC, 7th September 2011

Can Cleverbot 'think' like a human?   Daily Mail, 8th September 2011

Has inventor made a computer that's as clever as a human?   The Independent, 8th September 2011