After practice rounds and plenty of hype, Watson, IBM’s latest supercomputer, stepped up to the podium to take on two of Jeopardy’s greatest champions: Ken Jennings, who won 74 straight rounds of Jeopardy en route to becoming a pop culture icon, and Brad Rutter, who netted more money than any other contestant while winning the three of the biggest Jeopardy tournaments (defeating Ken Jennings in the process).
How did it go? Well, the tournament isn’t done yet, since it’s a three-day battle between the contestants. But after the first day, Watson has won $US5000, tying it for first place with Brad Rutter. Ken Jennings is in third place, with $US2000. How can a computer do this, you ask?
What Is Watson?
On stage, Watson is an portrait-oriented LCD monitor with an avatar consisting of a sphere-like graphic. In actuality, it’s housed in another room next to the Jeopardy stage, where it takes up the space of a spacious room, surrounded by refrigeration units. Watson is comprised of 10 server racks, which house 100 IBM Power 750 server units. He’s as powerful as 2,800 “powerful” computers put together. It is also equipped with 15 terabytes of onboard RAM.
How does Watson work?
Well first and foremost, Watson is not connected to the internet. So any knowledge base he draws from was preloaded by an IBM engineer. But that’s not what gives Watson the advantage, his two competitors basically know everything. What makes Watson so special is the speed at which he can analyse a clue delivered in conversational English, and derive an answer to that clue using his analytical engine.
When Watson is given the clue via electronic text, it is run through a series of complex algorithms which pick apart keywords, the relation of those keywords to each other, and the structure in which those words were used. From there it begins an association process where it generates and eliminates possible answers based on those keywords. It will also take into consideration previous clues and responses from the same category.
When Watson is working on an answer, his avatar will spin around, and the harder it works, the faster it spins. When it comes time to chime in, it delivers the answer in fluent English. When it is wrong, Watson will change colour and dim out, expressing digital shame.
Over the course of the episode, Watson proved to be mostly unflappable, answering questions with in a calm, methodical manner. text clues would be delivered electronically to Watson while Alex Trebek would read those same clues aloud to the other two contestants. As previous posts have laid out, Watson has light sensors that pick up the off-camera indicator to let contestants know when they can buzz in and respond.
When Watson would buzz in first, Jeopardy would show the top three answers (technically questions) Watson was considering, and how confident it was with each one (in percentage form). When it came time to answer, it would reply in a monotone manner, delivering the correct answer more often than not.
But though Watson has an encyclopedic knowledge, and seemed to display superior “reflexes,” it wasn’t perfect. In fact, it had a stretch where he responded incorrectly to three consecutive clues (even repeating Jenning’s incorrect response once). Here’s what IBM had to say about it:
One of Watson’s misses highlighted how difficult it is for a machine to play Jeopardy! The category was Olympic Oddities, and the answer was a gymnast with an unusual physical feature. Ken Jennings said “arm” and was wrong. Watson said “leg,” but host Alex Trebek ruled him incorrect because he didn’t say the gymnast’s leg was missing. During a viewing of the tape of the show earlier today, David Ferrucci, the IBM manager who heads up the Watson project, explained that Jeopardy! is an enormously broad domain of knowledge, and some for the classifications are pretty vague. In this case, the computer very likely didn’t understand what an “oddity” is. “The computer wouldn’t know that a missing leg is odder than anything else,” said Ferrucci. Still, over time, by reading more material and playing more games, Watson could come to understand. (Would he be able to understand that he is an oddity, I wonder.)
All that said, Jennings looked visibly frustrated by the end, hopefully foreshadowing a battle that should only intensify over the next two evenings as it reaches its final, climactic conclusion.
So who are you pulling for in this epic battle: man or machine?