RSS Feeds
Cameras
ZCam 3D Camera Is Like Wii Without Wiimote and Minority Report Without Gloves
Posted by Jesus Diaz at 1:10 PM on December 16, 2007
The ZCam is the first low-cost, consumer videocamera that can capture video with depth information and probably the first real challenger to Nintendo's Wiimote: with its 3D capture abilities it will allow you to play Wii-style without using any controls whatsoever. In fact, it is so precise that it will even recognise your finger gestures to fire a weapon or manipulate your computer like in Minority Report, but without gloves or any other external device:
The camera has sensors that are able to measure the depth for each of the captured pixels using a principle called Time-Of-Flight. It gets 3D information "by emitting pulses of infra-red light to all objects in the scene and sensing the reflected light from the surface of each object." The objects in the scene are then ordered in layers in the Z axis, which gives you a grayscale depth map that a game or any software application can use.
According to manufacturer 3DV Systems, the depth resolution is quite good: it can detect 3D motion and volume down to 0.4 inches, capturing at the same time full color, 1.3 megapixel video at 60 frames per second. While there have been professional cameras with depth capture in the past, this is the first time that a device of such characteristics is cheap enough to be built into any game system or computer.
As you can see in the video, the ZCam is completely different from the EyeToy or any other normal 2D webcam. Even while it's not as precise and flexible as a real full 3D motion tracking system, the videos show that it can indeed provide with a new level of interactivity in video games and any application—like computer user interface manipulation.
While the results could be quite impressive, I'm not convinced about some of the applications. The flight simulator, for example, seems to work great. However, unlike boxing, I have the feeling that I wouldn't be able to control a plane without actually grabbing something. Of course, I can pick anything to give me that feeling, just like a Nintendo Wiimote. But then again, in flight simulator games you want an actual joystick, so the "phantom" feeling of not having force feedback is the same for the ZCam and the Wiimote. For any other game, like first person shooters, this kind of technology could be really good if it lives up to its promise and developers can fully exploit it.
Its virtues could be even easier to apply in computer applications. The demonstration in the video, with the guy manipulating Vista with hand gestures, gives you a very good idea. As I use my iMac 24 to type this, I wish I can just wave my hand in the air quickly to consult a PDF, like Minority Report's Tom Cruise but without jumping in the sofa and scaring Oprah. Or touching, moving and clicking the mouse.
Since the ZCam is a piece of circuitry that can be integrated anywhere, and without taxing the CPU, I don't see why companies like Apple or Dell wouldn't adopt them for future desktop and portables. However, the technology has just been officially announced this week and it's too soon too tell. Whatever happens, it's good to see such a device coming to the market. Hopefully, we will see it in action with real world applications soon. In the meantime:
Comments (AU Comments · US Comments)
There are currently no AU comments for this post.
robbo
Posted 4:25 PM 16/12/07
god, my spelling sucks
robbo
robbo
Posted 4:25 PM 16/12/07
Wasn't the ZCam originally developed for use in broadcast as an alternative to chromakey - giving a studio the ability to replace backgrounds without having to worry about having a blue screen and proper lighting - just using the depth information? If I recall, it was created by an Isreali firm and shown at NAB for a while and although people were impressed with the concept the overall "look" of it was still very rough and chunky, with hard jaggy edges. I'm wondering if this is the same product but just repurposed for the game/PC market? Curious.
robbo
Spaceboy
Posted 2:31 PM 16/12/07
I can see totally see this being used -especiallyu with a VR device or even trackir- for a completely real flight simulator experience, actually not for the joystick and throttle controls which can still be real physical objects, but rather for all the switches, radios etc.
In fact, I'm happy I haven't invested more than 100 dollars so far in my home cockpit :)
Spaceboy
Sqube
Posted 1:33 PM 16/12/07
I'm far too cynical to believe that this is going to happen just because you've got some video.
It'd be even more awesome for boxing games than the Wii (which still hasn't released Fight Night Wii Edition, you stupid shits); there are definitely a lot of genres that I'd like to have something in my hand for.
Also, I'm waiting to see how FPS games are implemented. Basically, I'm waiting for something more than proof-of-concept.
Sqube
niravabhavsar
Posted 12:42 PM 16/12/07
I just hope this is not a prank. The video doesn't seem too professional.
niravabhavsar
niravabhavsar
Posted 12:41 PM 16/12/07
It is a very promising technology. I could imagine it replacing keyboard and mouse over next decade. It takes computer/video game interface to a whole new level. Hope this makes its way into the mainstream pretty soon.
niravabhavsar
d_saum
Posted 11:18 AM 16/12/07
Oh.... and "The Film Crew" FTW!!! MST3K lives!
[www.filmcrewonline.com]
There's also "cinematic titanic" with Joel and the VERY original crew, but I havent seen any of it yet.
d_saum
Nerdelphia
Posted 11:16 AM 16/12/07
This technology seems like something that could run parallel to the potential touchscreen technology has. That is, if there's something that could utilize this 3D cam techology the same way the iPhone/iTouch uses multi touch then happy happy joy joy.
Well, someone other then Nintendo that is, then again they seem to be doing something moreso in line with succeeding where the Power Glove failed.
And jesusdiaz: I thought it sounded more like they was saying "MASTER NINJA THEME SOOOOOONG!!".
Nerdelphia
d_saum
Posted 11:15 AM 16/12/07
I Love MST3K... watching "Night of the bloodbeast" right now. Jesus.. can you send me the mp3 of Master Ninja Theme song? Or tell me where you got it from?
I prefer Mike over Joel, but only barely. Mike just seemed to be funnier and had a better voice.
d_saum
jesusdiaz
Posted 10:37 AM 16/12/07
@strider_mt2k: I liked Mike too.
jesusdiaz
strider_mt2k
Posted 8:53 AM 16/12/07
I have to agree, the music didn't quite fit.
Don't sweat it, your average for awesomeness hasn't dropped by that much, Jesus.
Excellent concept.
Joel and the bots = Awesome
Mike and the bots...not so much.
strider_mt2k
jesusdiaz
Posted 8:42 AM 16/12/07
@pizzlepaps: I added the music. The videos were silent.
jesusdiaz
jesusdiaz
Posted 8:41 AM 16/12/07
@flumbo: I think the music is great, as it fits the movements and cheap look of the demo videos. As for not being useful ergonomically, I don't agree. If it works like the say and looks, I think it would be great in a wide range of situations.
Gaming to start with, but think about home theater setups. Be able to control your TIVO moving your hands. And I would to be able to casually wave my hand to have something happen in my computer.
Oh, and MASTER NINJA SIMPSON!
jesusdiaz
alin0steglinski
Posted 8:39 AM 16/12/07
@alin0steglinski: *cue minority report music*
alin0steglinski
alin0steglinski
Posted 8:39 AM 16/12/07
i want this for my mac so i can do everything like minority report!!!
alin0steglinski
howdytex
Posted 8:31 AM 16/12/07
The music cracked me up. I was expecting the camera to pan to the other side of the room and see a couple of guys making the noises, er, music.
howdytex
jesusdiaz
Posted 7:33 AM 16/12/07
@takemetoyourtoaster: they haven't mentioned the price. Their technology is available for OEMs, apparently. But according to them, low enough to be marketed cheaply in the mainstream.
jesusdiaz
hanswurst0815
Posted 7:31 AM 16/12/07
Now I'm watching this just to hear the music everyone's talking about. And I like it, very classy. But distracting non the less :D
hanswurst0815
jesusdiaz
Posted 7:05 AM 16/12/07
@altus: Where's is RealityFusion?
jesusdiaz
jesusdiaz
Posted 7:03 AM 16/12/07
@KenK: hahahaha, yeah. I'm retarded. And spanish. It's the way it sounds to me.
jesusdiaz
Wwhat
Posted 6:49 AM 16/12/07
If others did it in 2000, and nobody heard about it, and no products are available, then guess who did the bad job?
Wwhat
altus
Posted 5:54 AM 16/12/07
Guys, in 1997, RealityFusion launched something way ahead of this. The person behind this was Barry Spencer. Great innovator. All these guys are just copying decade old technology and ideas and not doing a very good job at it....
Take a look at this link from 2000:
[tinyurl.com]
altus
Wwhat
Posted 4:53 AM 16/12/07
0.4 inch eh, why not just say 1 centimeter.
I really like that this is possible now, and when they can increase precision you could use it as a 3D scanner for raytracing, or for simply putting your face on game characters too, this has lots of uses and potential.
I just hope and pray they specify in their rights the technology can't be used by cops/homeland security and such.
IF it is all real.
I guess the jpeg group has to start adding Z depth to their specifications now
Wwhat
pizzlepaps
Posted 3:22 AM 16/12/07
the demonstration is ruined by the cacophony in the background. the music is complete shite and sounds awful. what were they thinking
pizzlepaps
skulldriveshaft
Posted 3:14 AM 16/12/07
inquiring about SDK and cost right now :]
skulldriveshaft
peterlarson233
Posted 2:55 AM 16/12/07
i loved the music. and this is definitely something i would pay way too much money to have. imagine the possibilities. especially in applications like 3d modeling. it would be great.
peterlarson233
Arsenal6
Posted 2:40 AM 16/12/07
lol the music was gay...the preview was interesting
love 2 see wii with that...infact they had it at a playdium b4 with tekken motion tracking very fun
Arsenal6
TheTheTheTheWhat
Posted 2:24 AM 16/12/07
Ahaha, I remember when Kotaku posted that some exec saw the future of the Wii to be about cameras tracking movements and all of the comments were like, "WTF EYETOY EYETOY DUH LOL"... this is totally one of the greatest innovations I have seen in a lone time.
TheTheTheTheWhat
Set to evil
Posted 1:27 AM 16/12/07
Mimes of the world unite! Our time is now!
Set to evil
karnak
Posted 12:58 AM 16/12/07
'The objects in the scene are then ordered in layers in the Z axis, which gives you a grayscale depth map...'
I guess all the beenie weenie's caiming to have 10 inch cox can put up or shut up now. I think your IMing experience just got a whole lot more experiencable.
karnak
acemonster
Posted 12:50 AM 16/12/07
MMMMMmmmm Nicole Kidman. DAMN it I love redheads!
P.S. - My Gf is a redhead.
Oh yeah.. FUCKING cool camera!
acemonster
greyer
Posted 12:42 AM 16/12/07
[www.3dvsystems.com] It's photos like this that will kill this product.
greyer
NeoPoliticus
Posted 11:55 PM 15/12/07
Real-life technology keeps taking the fun out of things. The next thing you'll know, Darth Vader will get plastic surgery and cloned limbs.
NeoPoliticus
capitalass
Posted 11:55 PM 15/12/07
@KenK: Nice. MASTER NINJA SIMPSON. Lol.
capitalass
flumbo
Posted 11:20 PM 15/12/07
Sorry, Jesus. I thought the music detracted from the video. Love MST3K, but it didn't fit here...
I would love this camera on a laptop to manipulate screen windows. I wonder if they could have one that can also see your hands near the keyboard so small movements could be translated on screen. Looking at my 24" monitor in front of me it seems like it would be too much work to use as a main interface device on a big screen.
flumbo
itwasthedog
Posted 11:19 PM 15/12/07
all I could think about was how bad my hand would cramp after a zcam photoshop session. Remember that Tom Cruise was in good shape for minority report. Not because he trained but, because he had a cool full-body workout for a computer.
itwasthedog
randlsa
Posted 11:18 PM 15/12/07
@Terminator12139: Consider how dorky we look using our keyboard and mouse... But we've become accustomed to that "look".
randlsa
randlsa
Posted 11:17 PM 15/12/07
Fantastic! The possibilities are endless, but I wonder what the ergonomic fatigue factors are? It could be quite challenging to come up with a virtual control scheme that might allow game consoles to pick this technology up... Especially with the considerable durations of time some gamers spend gaming. As an example hold your arms out in front of you, and see how long you can do it comfortably.
randlsa
kamenwati
Posted 11:10 PM 15/12/07
That would be great for all aspects of computing and gaming....I can just imagine a Naruto game with it yeah judge me if you will I stand by my opinion
kamenwati
scram
Posted 10:59 PM 15/12/07
i somehow foresee this being implemented in a rpg for casting different types of spells..
scram
kinkypinky
Posted 10:28 PM 15/12/07
how much is it?????? cause that would be sick....
i see how much fun i could have with this...
lol
hide it behind a dvd box then u could
be able to control the computer or whatever... with "your mind controling powers"
kinkypinky
KenK
Posted 10:08 PM 15/12/07
Master Ninja Simpson? Are you retarded? It's the unofficial "Master Ninja" THEME SONG! Written by Joel and the bots from MST3K because the Master Ninja movie had no theme song of it's own. Simpson... really...
KenK
M. Schlabach
Posted 9:45 PM 15/12/07
I don't see this as a reality. I think it's photoshopped.
Dude! Think of the possibilities.
M. Schlabach
Saad Rabia
Posted 9:41 PM 15/12/07
Holly crap. What's up wit music?
And a great product for disabled people, not for regular people, yet.
Saad Rabia
yogibimbi
Posted 9:40 PM 15/12/07
the music is just great! Love it.
yogibimbi
takemetoyourtoaster
Posted 9:36 PM 15/12/07
when you say low cost how much does that mean really
takemetoyourtoaster
jesusdiaz
Posted 9:36 PM 15/12/07
@mcg1969: I put the music on the video. It's better than the original soundtrack (silence.) And dude, it's MASTER NINJA SIMPSON! Mystery Science Theater 3000. A must.
jesusdiaz
Terminator12139
Posted 9:22 PM 15/12/07
It's cool and works well, but I would feel like too much of dork moving my fingers and shit like that, like a little kid pretending he is in a airplane when in reality he is in a refrigerator box. It would also make you look like you are doing sign language or something.
Terminator12139
jesusdiaz
Posted 9:22 PM 15/12/07
@CoolZWhip: it has ALL to do with consoles. This device is targeted at game consoles, computers and other professional application.
jesusdiaz
mcg1969
Posted 9:21 PM 15/12/07
What the hell is with that music! Geez is there a NSWOD (Not Safe WithOut Drugs) label you can stick on there or sumthin?
mcg1969
CoolZWhip
Posted 9:19 PM 15/12/07
What the heck does it have to do with a video game console anyway?
CoolZWhip
dcartist
Posted 7:01 PM 16/12/07
Ear-bleedingly bad music aside...
It's a clever method of detecting depth information about visible pixels in the frame, to separate foreground and background objects.
The information that the camera ends up with, though, is more like a topographic map, than actually mapping 3 D objects, since you can't see behind objects, you can only see the depth of visible pixels.
According to this, the original application was "video-keying" without a monochrome background. i.e. greenscreen without the green.
[www.3dvsystems.com]
If they could improve accuracy, they could use this for orthotics manufacture (4 mm is not good enough).
Actually though, if you put down multiple cameras though, (synched for time windows on their infra red pulses to not interfere with the other cameras) you could use tomography to do a fairly good 3D, real time motion capture (at only 60 fps for now) without the annoying ping pong balls.
In my own field of medicine, using multiple cameras, I can see how this could someday be used for high quality kinematic capture in a gait lab for a very cheap price.
dcartist
dcartist
Posted 7:47 PM 16/12/07
Sorry...
dcartist
dcartist
Posted 7:46 PM 16/12/07
Robbo, I think this zcam is the same thing you're talking about.
[www.3dvsystems.com]
Videomatting without a monochrome background. Greenscreen without the green. It doesn't really make a 3D model, so much as a topographic map of all the visible pixels, showing distance from camera of every pixel, based on the timing of reflected IR on your sensor pixels.
The real key to using this as a computer control is coming up with a reliable, intuitive, OS that incorporates gestures in the air, no mean task.
With a bit of cleverness, definitely could do some great things with it.
If you set up several cameras at different angles, you can crunch a little math to do tomography, and do true, realtime motion capture without the "pingpong balls".
I work in pediatric rehab, and this camera might be really good for tracking dyskinetic and spastic movements to pull out meaningful gestures for communication. Also could probably use something like this in the gait lab as well, when the frame rate gets higher. The resolution isn't high enough to manufacture orthotics, but maybe then next gen could.
-
What is the core price of the chip since it's not mass produced yet? I suspect the D-channel detector chip has a much lower resolution than 1.3 MP... but the data is then mapped onto the 1.3M RGB visible pixels (figure it's not necessary to be that high in the D channel, because objects of interest are not going to be only 1-2 pixels wide).
dcartist
Vagabum
Posted 11:50 PM 16/12/07
And that music was way ultra annoying, no matter where it came from. Matisyaho those theme writers are not.
Vagabum
Vagabum
Posted 11:49 PM 16/12/07
Virtual sex just got 10 steps (or is that 6 inches?) closer...
The PC manufacturers will never adopt and bundle it until there is a slew of software that supports it and the software developers will never support it until there is substantial market penetration that will not happen because the PC manufacturers will never bundle it. Sony and MS may be all over it though as they struggle to catch up to Nintendo in the interface innovation category in hopes of attracting those elusive new casual gamers that are flocking to the Wii.
Vagabum
EMoShunz
Posted 8:12 AM 17/12/07
look for this, and other innovative technology coming to an appletendo wii2, november 2012.
EMoShunz
awdark
Posted 1:25 PM 17/12/07
Man if the youtube link was easy to click I would have given them 1 star just for the "music".
awdark
Geist420
Posted 3:47 PM 17/12/07
Might have been cool, but that "music" is horrid, closed the window.
Geist420
tchangtx
Posted 4:40 PM 17/12/07
Would love just to have this as a virtual keyboard. Typing in the air would be great.
tchangtx
toyotaboy
Posted 11:34 PM 17/12/07
I figured this was just going to be another eyetoy.. I'm mildly impressed.. forget wireless, go controller-less! Nice touch of using the MST3K master ninja song.
toyotaboy
Bokusatsu_Tenshi
Posted 5:58 PM 19/12/07
I'd have to agree with people saying the music is funny, but just don't fit the product.
Interesting concept though. But I'll just wait to see how they're gonna use it. Gestures are not enough to make it more than just a gimmick... like AR.
Bokusatsu_Tenshi
warrenE33
Posted 8:39 PM 18/12/07
this is all about the price. which i've not yet seen noted anywherrrreeee. if it's like 20 bucks, then it might show up in a console. if it's "merely" say 150 bucks then there's no chance.
has anyone heard of a price?
The tech is improved on eyetoy (and web cam motion sensing) of the past by mixing int he depth sensing. but depth sensing cameras have classically had a small range (like 3 feet where it's accurate, though you can set this 3 foot range to be out a certain place in the room - it doesn't necessarily mean their shit only works in the first 3 feet of their demo machine. that would of course be retarded for a living room gaming environment... curioussss.). The other problem classically has been the need to put out a lot of infra red light. like A LOT. And if you pump out a lot of infrared light, you don't want ot be staring into it for hours on end. it'd be like staring into a halogen bulb. except IR is invisible so you'd probably just get mysterious headaches every day.
still. i haven't used their zcam. fingers crossed that it is both improved on past tech and cheap. (a depth cam i experimented with a couple years ago, just black and white depth data mind you, ran more than 5,000 bucks). I'm nervous that they're planning to cheat on the zdepth by working in the color data as well (eye toy style). hmmmmmm.
warrenE33
focalproductions
Posted 4:08 AM 16/12/07
Now imagine!!!! interactive porn! how you gonna explain that to the wife if she ever walks in on your in the office
focalproductions
Dehappy
Posted 1:31 AM 16/12/07
"While there have been professional cameras with depth capture in the past, this is the first time that a device of such characteristics is cheap enough to be built into any game system or computer."
Any examples of these professional cameras? If I could find something like this, but usable with real lenses (and 1080i resolution) I would drop $10k without having to even think.
I've been doing greenscreen work as of late to support a media project. Green backdrop, green floors, lighting rigs, renting space, *tons* of manual post-processing, ... A true 3D camera would eliminate so much work for me! Record, apply a clip plane for depth and the floor, and I'm done. It'd even make it feasible for me to record out door scenes, which would be fantastic!
And I'm just imagining all the fun, fun experiments I can run with depth extracted from scenes, both for coding projects and robots applications! Robust depth exaction without stereographical guesswork? I could chuck all my computer vision books!
The console / gaming focus for this device seems... odd to me. I'm imagining a ton of more useful applications.
Dehappy
dale_massicott
Posted 10:25 PM 15/12/07
A camera is just a camera just like the eye is just an eye, its the software behind it that will work the real magic.
dale_massicott
james.ahlschwede
Posted 10:22 PM 15/12/07
This will probably make PlayStation partisans cry if it's really a Wii killer, but Sony demoed something exactly like this in one of business sessions at E3 2004, billing it as a possible EyeToy 2. There was a lot of buzz around EyeToy back then because it was the most successful console peripheral ever at that point. (This was pre-Guitar Hero.)
The proposed EyeToy 2 even used the same tech - pulses of infra-red light. I remember because the "how does it work" question came from Will Wright, and I think the guy answered almost out of reflexive respect before he realized that he might be giving out important trade secrets to a room full of strangers who weren't on NDA and might work for their competitors.
For whatever reason, the PlayStation Eye didn't end up using this tech.
james.ahlschwede