Steering a Balloon With Sound Waves Could Be the Secret to Touching Objects in VR

Steering a Balloon With Sound Waves Could Be the Secret to Touching Objects in VR

The ultimate goal to making virtual reality feel more like reality is letting users physically touch non-existent objects in a VR world. Until we figure out the technology behind Star Trek’s Holodecks, researchers from the University of Tokyo are experimenting with steerable helium balloons that VR users can safely reach out and touch.

What makes virtual reality so compelling: its ability to transport a user into any experience, it also what makes facilitating synchronised physical interactions in the real world so complicated. A system not only needs to be able to match the movements of a VR user’s hands, it also needs to be able to quickly move and position itself in a way that never obstructs, interferes, or causes injury to a user who has no way of seeing where it is.

Star Trek solved this program with hologram force fields that can appear and disappear anywhere in the blink of an eye, but that technology is firmly rooted in science fiction and centuries away from being a reality. So researchers at the University of Tokyo’s Shinoda & Makino Lab — possibly inspired by a birthday party — have developed a simpler solution built around a free-floating neutrally buoyant helium balloon.

In a paper published in December of 2020 for the Siggraph Asia conference, ‘Balloon Interface for Midair Haptic Interaction,’ the researchers detail an elaborate 360-degree rig that uses a series of ultrasonic phased array transducers to generate inaudible sound waves with enough force to push a floating balloon around and precisely control its motions. The setup also includes a pair of high-speed stereo imaging cameras to track the position of both the balloon and the position of someone trying to interact with it.

The resulting experience is a physical object that a user in a virtual space can touch and feel when trying to reach out and interact with a virtual object in VR. The balloon can be quickly repositioned so that it aligns with a virtual object and creates the sensation of a touch interaction. Even if the weight, firmness, and texture of the balloon itself don’t correlate to a virtual object (for example, the balloon isn’t going to provide any kind of matching resistance if someone is touching a virtual brick wall) the fact any touch is experienced should help enhance the believability of VR.

The other advantage to using a helium balloon is that if a VR user’s unpredictable actions result in them accidentally hitting or colliding with the balloon, nothing is going to get damaged in the process. It will simply be knocked away and the system will presumably be able to quickly reposition the balloon elsewhere. Were you to replace the balloon with a robot system designed to recreate a physical touch interaction there’s a safety risk to both the robot and the human user if they were to ever make accidental contact.

Will this approach give us real-life Holodecks before the decade is out? Not likely, but the use of imperceptible sound waves to move objects around without wires or other support structures is a clever use of existing technology. Whether or not the technology can be improved or upgraded to manipulate more than just objects that are lighter than air remains to be seen, but it’s another important step to making Holodeck technology more than just a plot device.