Facebook’s virtual reality headset, Oculus Quest 2, recently received a major software update, making it possible to create a mixed reality experience using the main body camera.
Mixed reality allows users to more actively participate in the mixed reality world by fusing the display of the real world and the virtual world. It can be said that it is a more advanced technology than virtual reality that displays only virtual reality and augmented reality that projects virtual images based on the real world.
Passthrough API Experimental, a new API introduced by Oculus, applies effects and filters to the real world, renders real images in virtual world games, and enables various applications. You can even draw in the real world with a virtual pen. A demo of switching between real and virtual world transparency with a slider was also shown.
Not only that, but you can also bring a real keyboard and type. This feature can be helpful for business use such as office and field work.
Applications using the API will not be able to access photos or videos taken with the Oculus Quest 2, Facebook said, meaning that user data is only processed on the device. According to Facebook, the API does not work on the early Oculus Quest and only targets Oculus Quest 2. The API will be provided as a software update in the future through the game engine Unity.