Gadgets
Wii Headtracking Creates 3D Window Display
Posted by Mark Wilson at 2:07 AM on December 23, 2007
Johnny Chung Lee is the PhD student from Carnegie Mellon University who has been rocking Nintendo fanboy hearts pretty hard by making the Wiimote do some spectacular feats. We first saw him track his fingertips on the screen for a "Minority Report" type of interface. Then he created an interactive whiteboard. Now, he's managed to create a headtracking simulation through the Wiimote, creating an amazing 3D window on the world.
The mod requires you to wear the sensor bar (or any IR-emitting headband, Lee makes some sweet goggles) and place the Wiimote by the screen (it's hooked up to a laptop with a TV-out for this demonstration). Essentially, your head becomes the mouse peeking through a 3D room, and the gameplay implications would be incredible.
Anyone else thinking that Nintendo should pay this guy whatever he wants to make some awesome games? Or, you know, Sony and Microsoft could always grab him up, too. These ideas just use the Wiimote's IR—technology that isn't exactly cutting edge stuff.

Comments (AU Comments · US Comments)
There are currently no AU comments for this post.
davemak
Posted 10:51 AM 22/12/07
A boxing simulation game would be awesome with this thing. Bob and weave.
davemak
LoganSix
Posted 10:46 AM 22/12/07
This guy is awesome. And I really like how well he does his presentations.
So, when he did the football stadium demo, how many of you were thinking back to when you were little trying to see what the rest of the shower looked like in the shampoo commercials? Come on, you know you were trying to look.
LoganSix
liquidsoapdispenser
Posted 10:43 AM 22/12/07
Wow, I never would have guessed the 3-D head tracking would look so amazing!
liquidsoapdispenser
shoelessone
Posted 10:42 AM 22/12/07
I love that guy.
That guy is so awesome.
shoelessone
mofojojo
Posted 10:40 AM 22/12/07
Damn, somebody, somewhere, get this guy a JOB!!!
mofojojo
weatherman
Posted 10:27 AM 22/12/07
Hell yes. Well, Actually, they should just pay him some kind of bounty for thinking of a way to use the existing Nintendo hardware to add a whole new level of interaction to their system. The game makers should be left to make the games.
I think this idea will get snapped up pretty quick by game developers. All you need as an accessory is those glasses, which would cost about $3 to make. Add a little mount for the Wiimote in front of the TV and you're pretty much ready for 3D gaming.
Nice work, Chung. Well thunk, and nicely presented.
PS I love the look at 3:53
weatherman
TimurY
Posted 11:45 AM 22/12/07
Wow. just wow. Want!
TimurY
xrikazen
Posted 11:42 AM 22/12/07
Nice work! How funny would it be if M$ hired him to work on a new xbox interface... =P
xrikazen
P3nnst8r
Posted 11:15 AM 22/12/07
@Seiven:
Rescue 911 i think. And i used to get such a workout from that game from constantly moving. I think these type of innovations are not only more fun and immersive, but you can get a pretty good cardio depending on how active it is.
Gjob Johnny!
P3nnst8r
Mandatory_Field
Posted 11:15 AM 22/12/07
I especially liked the reference to Procrastineering. I was going to study that....
Mandatory_Field
James
Posted 11:15 AM 22/12/07
I'm glad I watched this vid. Nice.
James
Inaki
Posted 11:14 AM 22/12/07
HE is actually the best geeky presenter I've seen. He's explaining things in a really clear way, awesome.
I'm not a huge gamer, but I want this!
Inaki
AlisdairC
Posted 11:12 AM 22/12/07
Outstanding stuff.Someone throw a load of money at the guy to develop ALL of his projects...
AlisdairC
Seiven
Posted 11:10 AM 22/12/07
@weatherman: 4:26 he looks like a kid on Christmas eve asking to open presents...lol...I want some games too.
Seiven
Mandatory_Field
Posted 11:09 AM 22/12/07
@davemak: Bob, weave and hurl (from the motion sickness)....
Mandatory_Field
Seiven
Posted 11:07 AM 22/12/07
that guy is my hero...I can't wait to see a rail shooter with headtracking. wasn't there a Police game that you could duck behind things at Dave & Buster's? I forget what it was called but I loved that game.
Seiven
Kinglish
Posted 1:00 PM 22/12/07
Nice job bro!! Keep up the good work!
Kinglish
mongchacha
Posted 12:59 PM 22/12/07
this really was great!
and i just want to echo others' comments about the clarity of his presentations.
mongchacha
mechanisma22
Posted 12:54 PM 22/12/07
Head tracking could make so much money, just have to make sure the not to infringe on patents. Like all those crazy football fans would be walking around a TV to see the game if the game was broadcast in 3D and they had a set-top box that interpolated the 3D with your head movement. Dr. Johnny Lee FTW!
mechanisma22
titan001
Posted 12:53 PM 22/12/07
Slanderizing. Stupid spell check.
titan001
titan001
Posted 12:53 PM 22/12/07
Wow, I think this is the first post I've ever seen without any slenderizing in the comments! Way to bring world peace, Johnny!
titan001
senator133
Posted 12:47 PM 22/12/07
I have to agree with the rest of you. Im amazed at how he spawns these inexpensive mods that result in extraordinary ideas. Johnny you have also earned a bookmark from me today.
senator133
hnkelley
Posted 12:26 PM 22/12/07
I've already watched his other presentations and bookmarked his website. I hate fanboys, but I just might have to nominate myself as an official Johnny Chung Lee fanboy! Great presenter, great ideas! My eyes about popped out of my head when he turned on the head tracking for the camera.
If I had the money, I'd love to try to mix all three of his Wii concepts into one set of toys. Would the universe implode? Or has this guy created the next, best computer-human interface set of tools?
hnkelley
dead_red_eyes
Posted 12:22 PM 22/12/07
Johnny is the man.
dead_red_eyes
senorbelly
Posted 12:15 PM 22/12/07
I love you, Johnny Chung Lee. You hear me? Johnny?
senorbelly
Maize
Posted 12:12 PM 22/12/07
This is pretty much exactly what I was doing as a research assistant in a University lab in 1997, only obviously not using a Wiimote. I used a headtracking rig that we called "The Bird". Our lab was looking into all kinds of aspects of this, and I only did one small part -- researching whether having realistic "head as head" movements worked better for people than having mouse-like "head as hand" manipulation movements. It's neat to see this coming into the mainstream, and I'd love to see his idea developed as a real product. One of the other students working in the lab did some headtracking work without using "The Bird", but instead using a webcam and doing face detection and then locating the person's eyes to use as the two points of reference. You could totally do that with a Playstation Eye or something.
Maize
Nakamuri
Posted 12:10 PM 22/12/07
This is absolutely amazing... I hope to see some games with this soon...
Nakamuri
Thom
Posted 11:56 AM 22/12/07
Very impressive, but I kept wondering when Kermit was going to come out from behind the scenes.
Thom
Reilaos, Original Asus Fanboy
Posted 2:15 PM 22/12/07
He should actually see if he can pitch this stuff somewhere...
Reilaos, Original Asus Fanboy
duroc
Posted 2:09 PM 22/12/07
I don't own a wii but, wow, I agree with everything you guys have said. Nintendo should totally hire/send money to that guy. that presentation was professional quality.
duroc
scotth0
Posted 2:08 PM 22/12/07
That guy is a studnerd. Way to go dude.
scotth0
Soulxside
Posted 1:40 PM 22/12/07
I came to comment that someone should throw money at this guy, but I see this has already been said many times. If I had a million dollars to invest in something, it would be him.
Soulxside
FrankenPC
Posted 1:40 PM 22/12/07
He sounds like a younger version of Dr. Farnsworth.
That is coolest use of 3D glasses I've ever seen. Yeah, it sucks to be the second person in the room watching this go down, but it's worth the wait. Wow!
FrankenPC
dcartist
Posted 1:21 PM 22/12/07
Absolutely EXCELLENT.
I thought the other stuff you did was pretty cool, but this last one tops it all.
I can see actually selling LED glasses and a mount for the wiimote, to do a game like this...
I also see using this as a way of allowing a person with a cognitive delays learn to interact with augmentative devices.
Too, too cool.
dcartist
dcartist
Posted 3:22 PM 22/12/07
An important point to remember here is that even though Wiis are generally hard to find and sold-out...
You only have to buy a $40 Wii-mote and use his software on a PC, without buying a Wii at all. The stores have plenty of Wii-motes around here.
dcartist
Robotube
Posted 2:55 PM 22/12/07
Johnny, as soon as you are awarded your PhD, I'm going to refer to you as Dr. Awesome.
Or do you already have it, Dr. Awesome?
Robotube
theorie
Posted 2:52 PM 22/12/07
what a gangster.
theorie
DaveTyranham
Posted 4:19 PM 22/12/07
Voyer.
DaveTyranham
branchclarke
Posted 4:58 PM 22/12/07
This guy is so impressive. I hope he is in some shape or form going to be bringing me fantastic technology in the near future. Nintendo needs to hire this guy for their next system NOW.
branchclarke
insomniac
Posted 4:55 PM 22/12/07
Now That's an innovation! can't wait till devs start to use this, along with more accessories for the wii!
insomniac
VishusBurn
Posted 7:47 PM 22/12/07
@dcartist:
Indeed. The only way this is Wii related is because he happens to be using the IR hardware out of a Wii-mote and sensor bar. Its still very impressive work he is doing, but i don't think he could ever get it to work on a Wii.
VishusBurn
pevans34
Posted 10:50 PM 22/12/07
I CANT WAIT for this level of immersion in video games. Once this hits mainstream Im definetly committing to that 60" plasma screen mmmmm
pevans34
RedBeard
Posted 10:33 PM 22/12/07
all i can say is wow....you're a genious...
RedBeard
keyshey
Posted 11:07 PM 22/12/07
this guy is so awesome!
keyshey
MadModderX
Posted 12:42 AM 23/12/07
This guy is awesome in the sense that he's taking a backwards approach to "normal" gaming. The only problem with using such an awesome feature in a game is two wiimotes, one game, a set of goggles, and a Wii. it could be considered "a little much to ask from a average player" for a wii game developer's stand point. Kudos on an amazing use of the wiimote, but unless its implemented in to the actual body of the console it seems to be a little much.
MadModderX
pj737
Posted 12:37 AM 23/12/07
If I was gay I would so do you.
pj737
skulldriveshaft
Posted 1:43 AM 23/12/07
Nintendo designed the Wii Remotes so efficiently and combined IR detection and bluetooth - just amazing by itself already.
Johnny, thanks for thinking outside the box man, I wonder what you're going to do when the Wii Fit gets released.
skulldriveshaft
VishusBurn
Posted 6:25 AM 23/12/07
@skulldriveshaft:
I don't think his skill has anything to do with the "efficiency" of the Wii-mote, but I would love to see what would happen if some investors gave him some fundage and worked with him to make third party hardware+software for the Wii, PS3, or the 360.
That would be neat.
VishusBurn
Churrito
Posted 6:57 AM 23/12/07
This guy is a genius.
Churrito
BigViper
Posted 10:16 AM 23/12/07
drools.....
BigViper
Klappstuhl
Posted 12:28 PM 23/12/07
Imagine a Starfox-Game with that!
Klappstuhl
uomolinux
Posted 12:16 PM 23/12/07
Just one word : Amazing
Nintendo, take note.
uomolinux
karnak
Posted 1:52 PM 23/12/07
"Chung Lee is the PhD student..."
Hey, CMU. Give this guy his frickin' Doctorate already. While your at it, be smart, give him his Associate Professorship, and a building!
Think his class would be full? Overfull.
karnak
deoki
Posted 1:25 PM 23/12/07
You can all help Johnny out by donating some coins for his projects, as you can see on:
[procrastineering.blogspot.com]
I will help him out. I'm sure and hope that many more will follow.
THIS GUY DESERVES A BIG CHECK!
deoki
macrumpton
Posted 5:36 PM 23/12/07
If you used two pairs of those lcd shutter glasses and a linked location bar switcher to alternate between two points of view you could have two people use the system, both seeing their own perspective (or even have them see two entirely unrelated images). This assumes the wii has the ability to handle the computations for both players, or maybe it would be just easier to sync two wii's together, and have each one dedicated to each player.
macrumpton
MacAddict21
Posted 7:32 PM 23/12/07
greatest geek EVER
MacAddict21
nwgrede
Posted 11:13 AM 24/12/07
"Hello patent office?"
"yesh?"
"My name is Johnny Chung Lee and have I got a deal for YOU!"
nwgrede
JerryMandering
Posted 9:04 PM 22/12/07
No matter if he can get it to work on a Wii or not, you have to admit that his presentation just completely changed the way most developers are going to develop games from this point forward.
Johnny Lee just changed the way we are all going to be playing games from 2008 on.
Good work Johnny! I love this guy... in a non-homo way of course. Sony and XBox both need this tech more than Nintendo. Hopefully they pay him sh!tloads of money.
JerryMandering
ACTUALLYMAYBE
Posted 11:33 AM 22/12/07
he deserves money NOW
-6 high definition projectors for all the walls of a room
-crysis
hey remember that virtual-fantasy-fulfillment room in minority report?
ACTUALLYMAYBE
GatorGiz
Posted 11:33 AM 22/12/07
They need to make a condom attachment with infrared. Whoa, think of the possibilities.
GatorGiz
Abduction
Posted 10:59 AM 22/12/07
Holy damn!! In answer to the question in the post; I think that the first game company to see this should pay this kid ridiculous sums of fat cash to develop games for them.
Abduction
toyotaboy
Posted 9:47 PM 24/12/07
this guy is a genius.. I hope nintendo is listening, having a truly immersive viewpoint changing display would knock every other system out period.. or nintendo would be making a killing on patent rights.
toyotaboy