Kinect 2: So Accurate It Can Lip Read?

Since its launch in 2010, the Kinect has changed gaming and inspired a whole new genre of hacking. But Kinect 2 might blow it out of the water, if rumours are to be believed.

According to Eurogamer, Microsoft plans to release a second-generation Kinect device that will offer improved motion-sensing and even voice recognition.

How much more accurate will it be, exactly? Well, a source told Eurogamer that it should be able to lip-read. If that is true, that is wild. The same source also claims that it can track the pitch and volume of player voices and facial characteristics to measure different emotional states. That definitely sound plausible, though we’ll have to wait and see if Eurogamer’s reports turn out to be correct.

As well as some pretty serious image-processing software, I’d expect we would see a pretty hefty improvement in Kinect hardware specs to achieve these kinds of results. Especially when it comes to the CCD, which currently manages a paltry 320×240 pixel resoltuion. Whatever the spec, the new Kinect would probably launch along with the next-generation Xbox which, according to EDGE, could be available as soon as the end of 2012. [Eurogamer]

Image: bfishadow

Discuss

(38 Comments)
  • [–]

    Trav

    Tuesday, November 29, 2011 at 9:08 AM

    My mother is deaf and wears hearing aids but her hearing is declining as she ages and hearing aids are no longer adequate. This article made me think if lip reading becomes possible with a new kinect then glasses with an inbuilt HUD showing real life subtitles cannot be far away. that would be awesome.

    • [–]

      Graham

      Tuesday, November 29, 2011 at 9:16 AM

      Thats an awesome thought, although its probably a lot easier to simply create a pair of glasses that has inbuilt speech recognition as I think it will allow for a much more compact unit and be able to detect conversations not in direct line of site of the wearer. Great thinking though.

      • [–]

        Adam

        Tuesday, November 29, 2011 at 12:07 PM

        Voice recognition services don’t pick up some accents properly. I think the best solution would be a combination of both.

        • [–]

          Trav

          Tuesday, November 29, 2011 at 4:37 PM

          Yeah line of sight is a big drawback, but so is background noise, a hybrid of the two would be optimal

  • [–]

    Thorbjørn

    Tuesday, November 29, 2011 at 9:08 AM

    and I just watched 2001: A Space Odyssey last night… Lip-reading electronics are dangerous. Especially out in space.

    • [–]

      Michael

      Tuesday, November 29, 2011 at 9:54 AM

      So in space, they will hear you scream? I like this idea.

  • [–]

    steve

    Tuesday, November 29, 2011 at 9:38 AM

    But will it understand Aussie lips, or will we have to wait a year for an update?

    • [–]

      Roland

      Tuesday, November 29, 2011 at 10:04 AM

      lol yeah, work out “struth” or “crikey”

  • [–]

    cayal

    Tuesday, November 29, 2011 at 9:58 AM

    Two things:

    1. Kinect hasn’t changed gaming, it’s not even new technology.

    2. The purpose of lip reading is what?

    • [–]

      Matt L

      Tuesday, November 29, 2011 at 10:35 AM

      1. Kinect has changed gaming. I can’t remember using my body to control games before Kinect… Can you?
      2. Even if you don’t have a purpose for lip reading. Some people rely on it to communicate… But that’s not the point, the point is, if this device can read fast moving lips in detail, then its testiment with how well the unit can be used… For other small detail tasks.

      I was impressed by the tech in theory, but in practice, I found it less then ideal. They were originally going to bring Kinect out in HD. They really should have first.

      • [–]

        cayal

        Tuesday, November 29, 2011 at 10:40 AM

        EyeToy was used in the PS2 era. The exact samething.

        And I am sure that wasn’t even the first thing used.

        • [–]

          wsDK_II

          Tuesday, November 29, 2011 at 11:34 AM

          comparing the EyeToy with Kinect is like comparing an iPad with a PC – the iPad can do a few basic things, and is cheap crap, however the PC does everything really, really well.

          the Kinect does everything really well, and the EyeToy was more of a gimic

          • [–]

            cayal

            Tuesday, November 29, 2011 at 11:48 AM

            “the Kinect does everything really well, and the EyeToy was more of a gimic”

            No and No, they’re no gimmicks.

            And I am not debating this with an obvious illiterate fanboy.

            • [–]

              wsDK_II

              Tuesday, November 29, 2011 at 1:06 PM

              by replying you kind of are, btw.

              • [–]

                Lolz

                Tuesday, November 29, 2011 at 7:10 PM

                LOL The Wii is like 100 better at motion tracking than Kinect could ever dream to be. How does it feel knowing even Nintendo did something better than your pression RRoD?

                • [–]

                  Matt L

                  Friday, December 2, 2011 at 2:56 PM

                  Wii’s motion tracking is very limited, in that you need at least one control. Kinect can watch 4 peoples movements at once. Arms, legs, body, head…. Kienct 2 will bring in finer details, like fingers, feet, hands and of course, lips… The fact that this can read lips and you can’t is a clear indication of why this technology is more then just throwing a couple of accellerometers in a piece of plastic with some buttons.

      • [–]

        Cameron

        Wednesday, November 30, 2011 at 4:16 AM

        The playstation 2 had the eye toy which was basically the same thing as kinect. Just Kinect is a lot more advanced. What would really be cool is if we could use our body to control a character. Imagine running in place playing Gears of War.

  • [–]

    Norgan

    Tuesday, November 29, 2011 at 10:31 AM

    @cayal If Kinect hasn’t changed the face of gaming then what has?

    It’s an extension of the Wii sure, but it does so much more and no other company has come close to replicating the feature set, even though Apple may be trying to patent a similar device.

    The Kinect has changed how we interact with our devices and what it will bring with V2, Windows Phone interaction and the Kinect for Windows device will not only change gaming but it will change how we interact with all digital devices. How usefull or practical it may be is to be debated but it certainly is some fun stuff to use :)

    • [–]

      cayal

      Tuesday, November 29, 2011 at 10:50 AM

      The answer is simply nothing. The Wii, and by extension Kinect and Move have dumbed down gaming, not improved it. Nothing on the Wii is overly impressive, or new. It’s all been done before and generally that is in the arcades.

      They all suffer the same problems, lag (Wii is not 1:1, Kinect is certainly not, Move claims it is and is probably the most acurate) and lack of sufficient games (Move, Kinect) rather outputting shovelware (Wii, Kinect) or tacted on for no real reason or purpose (Move).

      As for Kinect, it’s not an extension of the Wii (the PS3 Move is). It’s a more advanced EyeToy which was released for the PS2 and probably that was used on PCs before that was released.
      This is where people are sucker into Kinect. It’s not new. Not even close. EyeToy was an interactive webcam with it’s own fitness games.
      The major problem with Kinect is that it will NEVER work without a controller. You are simply limited by how it works that it will never substitute a controller. Think about all the action you use when gaming – walking, running, jumping, climbing, shooting, reloading etc – you can’t do this in your living room without a controller.
      The tagline ‘You are the Controller’ should be amended to ‘You are the Controller…for shovelware games’.

      As for interaction with devices. It’s all been done before. Don’t buy into this Minority Report hype they keep spoon-feeding the masses.
      As for the Kinect hacks, yeah they look cool. Totally useless and can’t be done on a console for games. But still cool.

      • [–]

        Matt L

        Wednesday, November 30, 2011 at 9:34 AM

        I like to think of it in terms of what it could mean for Robotics… Sure, it’s a gimmik now, but with all these people able to work on it. This thing is basically the eyes and ears of a robot, it just needs to be trained – which is what’s going to happen now it’s open for any developer… Once Microsoft builds it’s database of what kinect can recognize, they’ll apply it to some pretty advanced tech… And I agree, EyeToy was also a step in that direction, but EyeToy was doing exactly what webcam games were doing before that – No difference AT ALL. Before eye toy, simple motion tracking was the step before that… It just continues to expand, and if u keep relating back to older tech, at the end of the day, you’re basically saying a T1000 robot is just an eye toy that does more. <- Which is only part truth.

  • [–]

    wsDK_II

    Tuesday, November 29, 2011 at 11:36 AM

    wow, ok – so you work for Sony then?

    Kinect DOES work without a controller, i have one in my living room that i hacked to work the lights, blinds, TV and occasionally games too :)

    i can control the volume of the TV by simply rotating my hand.

    give me ANYTHING else that does that, and at the same time,tell me it is useless.

    • [–]

      cayal

      Tuesday, November 29, 2011 at 11:57 AM

      Ahh the typical fanboy response: “You work for [rival company] then?”

      Did you completely ignore the fact I criticised all motion controls?

      I can control the volume of my TV by simply pressing a button, which happens to be much easier.
      As for hacking your lights, blinds, TV etc, two things:
      -I doubt you hacked it to be honest.
      -Centralising your living room into one remote has been done for years.

      • [–]

        Blake

        Tuesday, November 29, 2011 at 12:25 PM

        There is an extremely large difference between controlling something with a remote and controlling something with your body, in the fact that you don’t need a remote.

        • [–]

          cayal

          Tuesday, November 29, 2011 at 12:46 PM

          Indeed, except turning up the volume with your hand and controlling your living room with your hand are two entire different things one of which cannot be done.

          Kinect isn’t new technology, I don’t know why people think it is.

        • [–]

          Will

          Tuesday, November 29, 2011 at 1:00 PM

          Just ignore him. He’s clearly an idiot.

          • [–]

            wsDK_II

            Tuesday, November 29, 2011 at 1:10 PM

            ignore blake, cayal or me?

            in anycase, cayal seems to have a mission to payout the kinect wherever possible, an ‘anti-fanboi’ if you will.

            • [–]

              cayal

              Tuesday, November 29, 2011 at 1:40 PM

              A mission? Why because I am actually talking about the product rather than blindly praise it like you do?

              Instead of talking about me, talk about Kinect, or are you incapable of doing so?

            • [–]

              Lolz

              Tuesday, November 29, 2011 at 7:13 PM

              So seeing as you have admitted that you are against cayal, and that he is an anti-fanboy, you effectively admitted to being a fanboy? I knew it, you probably jizz in your pants at the announcement of kinect sports add ons.

            • [–]

              Damo

              Tuesday, November 29, 2011 at 7:24 PM

              So was why not publish some video of your hacked kinect magic eh?

          • [–]

            Bill S. Preston Esq.

            Wednesday, November 30, 2011 at 8:57 AM

            This. And a troll.

            • [–]

              cayal

              Wednesday, November 30, 2011 at 9:33 AM

              You still sooky because I called you out on your generalisation of Sony TVs?

              • [–]

                Bill S. Preston Esq.

                Wednesday, November 30, 2011 at 1:22 PM

                Not at all. =D
                I actually think its pretty funny! An advanced eyetoy!
                BWAHAHAHA!

  • [–]

    Penmonicus

    Tuesday, November 29, 2011 at 12:45 PM

    @Cayal: I was similarly unimpressed until my friends bought me one for my birthday. I figured it’d be pretty much the same as the EyeToy. It’s not. Its capabilities are staggering.

    • [–]

      cayal

      Tuesday, November 29, 2011 at 1:36 PM

      I said it is an advanced eyetoy.

      • [–]

        IJustKnowStuff

        Tuesday, November 29, 2011 at 4:45 PM

        As much as a single blade pocket knife is to a multi-tool army knife. But you are right, if you wish to ignore everything else about it. It is an advanced eyetoy. It just has more capability and functionality. Ignoring the same aspects that you are and you could as well say a gun [as a weapon] is just an advanced form of a thrown rock [as a weapon].

        • [–]

          cayal

          Tuesday, November 29, 2011 at 6:26 PM

          *sigh* Aside from idiotic analogies, you are saying what I am saying.

          To use your analogy (correctly), it is the advancement an assault rifle made on a hand-gun.

    • [–]

      Lolz

      Tuesday, November 29, 2011 at 7:14 PM

      I’ve used one, not even impressed. Pretty meh if you ask me, only people who get excited over it are casualfags.

  • [–]

    Patrick_Bateman

    Monday, December 12, 2011 at 5:12 PM

    I don’t care for motion gaming, nor do I care for someone who thinks that ‘hacking’ their Kinect & controlling their loungeroom is cool. It’s not. It’s lame.

    Console gaming is about sitting on your ass on the couch for hours on end with a controller in your hand, staring at a screen. Always has been & with me, it’ll always be that way.

Join The Discussion