The Magic Of Watson, IBM's Question-Answering Supercomputer

A computer that can have a conversation with you in real, human language is a hallmark of science fiction films, but has always seemed ludicrously unrealistic. Here's the thing: IBM just built one.

With the goal of creating a computer that can win at Jeopardy, a group at IBM has been hard at work on Watson for years. Jeopardy is the perfect challenge for a computer who is designed to really understand language. With its wordplay and puns built into reverse-questions, it requires a knowledge not only of facts, but of how language is used in the day-to-day.

In order to do this, Watson uses an insane amount of computer power—it is a supercomputer, after all—to analyse loads of algorithms at once.

Watson's speed allows it to try thousands of ways of simultaneously tackling a "Jeopardy!" clue. Most question-answering systems rely on a handful of algorithms, but Ferrucci decided this was why those systems do not work very well: no single algorithm can simulate the human ability to parse language and facts. Instead, Watson uses more than a hundred algorithms at the same time to analyse a question in different ways, generating hundreds of possible solutions. Another set of algorithms ranks these answers according to plausibility; for example, if dozens of algorithms working in different directions all arrive at the same answer, it's more likely to be the right one. In essence, Watson thinks in probabilities. It produces not one single "right" answer, but an enormous number of possibilities, then ranks them by assessing how likely each one is to answer the question.

Of course, understanding complex human language and being able to respond with it are two different things. But clearly, we're a lot closer to a HAL-style conversational computer than you might have thought. Crazy.

Be sure to check out the full story of Watson in the NY Times Sunday Magazine, and also try your hand at playing against Watson in a Jeopardy-style trivia game. Spoiler: he'll beat you. [NY Times]