The ultimate goal of bringing virtual reality closer to reality is to allow users to interact with objects in the virtual reality world. It is not easy to realize a device that creates a realistic virtual space like the Holodeck in the movie Star Trek, but the University of Tokyo research team manipulated it with ultrasonic waves that a virtual reality user thinks as touching an object when they reach out. I am conducting a balloon experiment.
It is to immerse users in all experiences or to realize synchronized physical interactions in the real world. It is necessary not only to match the object to the movement of the user’s hand, but also to move quickly to determine the position so that it does not interfere or injure the user.
According to a presentation at the SiggraphAsia conference held in December 2020, it is a study to generate sound waves to control the precise position by floating a balloon using an array of multiple ultrasonic satellites arranged at 360 degrees. It also includes a high-speed stereo image camera to track both the balloon position and the position of the person operating the balloon. It allows users in virtual spaces to reach out to realize the experience of touching and feeling virtual objects.
Balloons are quickly rearranged in line with virtual objects, providing a sense of touch and interaction. The fact that you can experience any tactile sensation even if the weight, hardness and texture of the balloon itself are not correlated with the virtual object helps to increase the reliability of the additive reality.
Another advantage of using a helium balloon is that there is no damage even if a virtual reality user hits a balloon due to unpredictable behavior. Not only can you tap the balloon to drop it, you can immediately reposition the balloon to a different location. If the balloon is replaced with a robot designed to reproduce physical touch interaction, there will be a safety risk if the robot and the user accidentally make contact.
Of course, it is unlikely that it will become a real holo deck in this way. However, the use of ultrasound to move objects without the need for wires and other supporting structures can successfully apply the existing technology. It is not known whether objects lighter than air will be manipulated through technological advances, but future research can be expected. Related information can be found here .