Microsoft announced Microsoft Mesh, a multi-reality platform, at Microsoft Ignite, an annual meeting of its developers held on March 3-4. In addition, in collaboration with Niantic, a developer of Pokemon GO, a demo video of Microsoft’s mesh version of Pokemon GO, where you can experience Pokemon roaming in the real world through HoloLens, was also released.
Microsoft Mesh is a multi-reality platform using Microsoft Azure. Microsoft Mesh provides applications such as multiplayer games sharing the world, remote work, remote medical treatment and events, education and research, where remote users gather in a multi-reality room to collaborate with each other. Features such as collaboration with real-time synchronization with the state using holoportation appearing in the other party’s space like a semi-transparent hologram, support for multi-devices that can participate with MR or VR headsets, PCs or smartphones, Azure Cloud-based session management and user management, etc. I take it as. Microsoft is also planning integration with its teleconferencing services Teams or Dynamics 365.
By sharing the platform, Microsoft Mesh makes it easy to enjoy conversations and share virtual reality spaces with people far away. Alex Kipman, who is called the father of HoloLens, says that being able to see the same hologram is magic. Microsoft Mesh also plans to implement an avatar generation function in the coming months. Using Microsoft Azure, spatial rendering and real-time remote sessions are possible.
Microsoft Mesh will support a variety of mixed reality or virtual reality headsets, including HoloLens, and will work on a variety of devices such as smartphones, PCs, and tablets. Third-party applications that support Microsoft Mesh are also being developed. For example, OceanX, a non-profit organization that supports ocean-related education, has announced a partnership with Microsoft Mesh and also released Holographic Labo, a Microsoft mesh-responsive app.
Holographic Labo can call marine life models and data in labs and offices around the world on board, and communicate in real time while sharing virtual reality space with other researchers.
In addition, Niantic, the developer of Pokemon GO, has announced a partnership with Microsoft with the aim of introducing new technologies to the boundaries of the digital and real world and providing experiences that connect people in a different way. As a proof-of-concept demo for consumers, Microsoft’s mesh version of Pokemon GO has been released.
When you raise your hand, the inventory is displayed in the real space, and when you throw a monster ball, Pakichu appears. It is also possible to give fruit to Pokémon, and in addition to Pokémon coming under your feet, you can properly recognize Pokémon led by other players. John Hankey, CEO of Niantic, said Microsoft’s mesh version of Pokemon GO will be a tool for understanding new augmented reality possibilities and future evolution in both software and hardware.
Of course, Niantic is also developing the Niantic Real World Platform, a platform for augmented reality apps. Although it has already begun providing it to the outside world, it did not disclose the specific details of the collaboration with Microsoft and the relationship with the company’s platform.
Meanwhile, during this event, Microsoft announced the Intelligent Speaker, a smart speaker for conferences that supports real-time translation and text messages.
The intelligent speaker announced this time is a device for Microsoft Teams, a company work collaboration tool that can make video calls and more. The built-in AI and 7-array microphone will automatically identify up to 10 participants to recognize speech as text, as well as support translation into other languages.
Regarding the issue of personal information protection for such character recognition and translation functions, it is said that automatic identification of speakers and automatic character recognition for a specific person can be turned off. The level of personal information and security is expected to be set by the user.
In charge of developing intelligent speakers, Yealink, a major Chinese telephone company, and EPOS, a sound equipment brand under Sennheiser, are responsible for the development of intelligent speakers. Although it did not mention the price and exact timing of release, Microsoft said it will be released in limited quantities in the second half of 2021. Related information can be found here.