Apple Multi-Touch Data Fusion Adds Camera, Voice, Force Sensors

Apple has been working in new multi-touch technology that combines touch interfaces with input from the camera and the microphone. For example: this will allow you to select text in the iPhone, say “copy,” go to another application and say “paste” to make this task really easy. The most intriguing part, however, is the use of a camera in laptops and desktops.

This will require two cameras, one for video chat and the other for the “hand reading,” but it opens a lot of possibilities. To start with, the entire keyboard can become a gesture control pad without even having to touch the surface. In addition to that, it can be combined with actual touch technology to identify single fingers on the surface, with the possibility of assigning specific functions to them.

The system even contemplates combining all this with accelerometers and force sensors, so the touch action can generate secondary data. One example of this may be applying a deformation effect to an image or a sound effect to a music track, giving it more or less strength depending on the force you use in your action. [USPTO via Unwired View]