Since Google acquired the artificial intelligence company DeepMind for $US628 million last year, it’s put the software to hard work… playing Atari 2600 video games. Really. It’s learning how to play 49 different Atari games showcases the promises — and the weaknesses — of DeepMind’s software.
A paper published today in Nature details how the AI learned to play the video games on its own. That may not seem like much of an accomplishment in a post Deep Blue-world, but DeepMind works very differently. It was never taught any rules of the games, so it had to learn by every step trial and error, essentially hitting keys by random until it stumbled upon the right move.
This works spectacularly for some games, like Space Invaders and Pong and especially Video Pinball, where the AI outmatched professional human game testers. But in 20 out of the 49 games, it was never able to get up to human skill level. MIT Technology Review’s Tom Simonite explains why:
The classic game Ms. Pac-Man neatly illustrates the software’s greatest limitation: that it is unable to make plans as far as even a few seconds ahead. That prevents the system from figuring out how to get across a the maze safely to eat the final pellets and complete a level. It is also unable to learn that eating certain magic pellets allows you to eat the ghosts that you must otherwise avoid at all costs.
DeepMind’s software is essentially stuck in the present. It only looks back at the last four video frames of game play (just a 15th of a second) to learn what moves pay off, or how to use its past experience to choose its next move. That means it can only master games where you can make progress using tactics that have very immediate payoffs.
Future versions of DeepMind will likely have more memory that allows it to make better long-term strategy decisions. And eventually, it could be applied to solve real world problems like understanding search terms and translating text. But, for now, rest easy. You can probably still beat a computer at Ms. Pac-Man. [Nature]