Friday, February 18, 2011

I'll take "The Humans" for $1,000,000, Alex. (The obligatory Watson post.)

Full disclosure: I worked one day a week for a year on improving Watson's speech synthesis.  While I think the voice added flair and personality to the exhibition, everyone knows that question-answering is what Watson is all about. And I didn't have any hand in that.

Wednesday marked the end of the three day exhibition Jeopardy! match between Watson, IBM's question answering system, Ken Jennings and Brad Rutter.  Watson decisively beat the human competitors.

There is no shortage of commentary on the implications of Watson's victory.   One of my favorite pieces was by Ken Jennings himself.  He's played more games of Jeopardy! than anyone else, and describes the experience in a very measured way.

One of his observations sticks out: "I expected Watson's bag of cognitive tricks to be fairly shallow, but I felt an uneasy sense of familiarity as its programmers briefed us before the big match: The computer's techniques for unraveling Jeopardy! clues sounded just like mine."

 It is in this that Watson represents a step forward for Artificial Intelligence.

There is a human exceptionalism that most people have when discussing "intelligence".  This exceptionalism leads to a dismissal of the success of Deep Blue's defeat of Kasparov as computation not intelligence.  It leads to descriptions of Watson as a "big database".  The same exceptionalism leads to the Turing Test as the final arbiter of intelligence.

Prediction: No machine will ever pass the Turing Test.

The test is explicitly defined to defend the exceptionalism of human intelligence.  It all but explicitly says, "humans are intelligent. To be intelligent, you must be indistinguishable from a human."

Before this test is passed, machines will continue to perform tasks which we consider to demonstrate intelligence better than humans can.  Chess: done.  Quiz shows: done. Mathematical theorem proving: done.  What's next? Perfect SAT scores. Perfect GRE scores?  Writing a five paragraph essay?  All conceivable in the next, say, five years.  Performing basic research and writing an accepted peer-reviewed publication of its findings.  Certainly further off, but not inconceivable.  But none of these systems would pass the Turing Test; people would still ask, "can it make me coffee?" or "can it tell me why Ronnie and Sam are still together?"  And yet, progress in artificial intelligence will be undeterred.

Ray Kurzweil's prediction of a self-aware system will almost certainly predate one that can convince us that it is human.  Despite its ability to describe its "thought" process, and outperform humans on tasks that were formerly considered representative behavior, there will be those that won't consider this system "intelligent" until it passes the Turing Test.

Watson's success comes from some very impressive machine learning and natural language processing, not only in its ability to generate answers quickly, but in its collection of knowledge.  The individual ml and nlp components used in Watson may individually be incremental improvements on previously existing approaches.  Yet this doesn't mean that Watson is itself an incremental milestone.  Watson bested a human at a task that we believed, until Wednesday night, required intelligence.  But more impressively, it won in a way that was similar (in spirit if not in hardware) to the way humans play Jeopardy!

Requisite CYA: As always, this post doesn't represent anyone's views other than my own.