It may have started out as a way to let players physically interact with their games, but the Xbox 360’s Kinect sensor has since developed a life of its own. Its clever combination of cameras and sensors have been embraced by hackers and researchers who’ve used it in countless project, including Microsoft’s own research division who’ve now found a way to use only the Kinect to perfectly track and mimic a highly-articulated human hand.
What makes the research particularly impressive is the Handpose’s ability to keep track of a hand and the position of all the fingers while the person or camera is moving about, and from distances where other systems would fail.
The obvious practical advantages to this system are as a radical way to change how players could interact with a game. They could physically move the pieces on a virtual chess board, quickly reposition troops in a real-time strategy game by hand, or even manually place Tetris blocks instead of waiting for them to fall. But Handpose could even be trained to read sign language, converting the hand motions into spoken words, and serving as a virtual translator. [Microsoft Research]