Technology that reflects the real world in VR in real time?

DreamWalker is a technology that allows you to explore a virtual reality space while moving your body by scanning a real space with a camera and directly reflecting it in the virtual reality space environment. This technology was announced by Microsoft during a global event (User Interface Software and Technology 2019) dealing with user interface related technologies.

Looking at the video related to Dreamwalker, at first glance, a man with something like a virtual reality controller seems to be walking inside an ordinary facility. However, what he sees in men’s sight is a skyscraper made of CG. Yellow arrows indicate directions, so you won’t get lost or hit a wall even if you can’t see the real world.

The realization of Dream Walker is the use of virtual reality headsets, smartphones and PCs with GPS functions. The headset is equipped with a virtual reality camera to shoot the surroundings and two RGB-D cameras to measure the depth.

The process of synthesizing the real world and virtual reality space is all three steps. After converting the image captured by three cameras into a 3D map with a difference in height and height, based on the 3D map, a place to walk or not to walk is distinguished and reflected in the virtual reality space. Finally, by placing objects, it compensates for the subtle differences between the real world and the virtual reality space and guarantees user safety. For example, if there is inconsistency in the virtual reality space, such as showing an unexpected path, it serves as a guide by displaying a mark.

In the virtual reality experience, the number of standalone virtual reality devices that operate alone is increasing in recent years because the code has limited body movement by connecting devices. However, virtual reality basically assumes that the user enjoys in the same place, so there is hardly any content such as virtual reality space exploration that moves around the real world. Dreamwalkers have the potential to bring innovation to various virtual reality contents by transforming the real world into a virtual reality space in real time. Related information can be found here .