Niantic launches AR development kit beta

Pokemon Go developer Niantic has launched a closed beta for Lightship, an augmented reality developer kit.

Niantic Lightship is a development kit that enables advanced augmented reality applications on smartphones. It was renamed from the former master, Niantic Real World Platform. In the previous alpha version, real-time mapping that captures the surrounding environment in 3D form, semantic segmentation that divides areas such as ground, sky, and buildings from camera images, and multiplayer that allows users to share AR experiences with low latency at the same time, etc. Several tools have been added, including enhancements in three areas. In the case of mapping, it not only recognizes the height of the ground, but also recognizes trees and buildings as a three-dimensional shape, enabling realistic reflection behavior on augmented reality objects.

The feature of the Niantic AR development kit is that it enables real-time 3D mapping on a general single camera smartphone without requiring special functions such as depth sensors and multi-cameras such as the iPhone 12 lidar. It uses technologies such as Matrix Mill, a computer vision startup acquired in 2018, and 6D.ai, an AR cloud company acquired in 2020.

Currently, mobile augmented reality technology is difficult to accurately grasp the three-dimensional shape of distant buildings and complex environments, and it is difficult to grasp something like a human even after knowing the shape. The AR development kit performs semantic subdivision so that this area can be identified as the sky and the space as a building, and it is possible to express virtual characters and objects by superimposing them.

Real-time multi-user is also one of the features of this technology. By combining local P2P communication and communication through a server, you can experience low-latency augmented reality with real-time multiplayer games. Niantic plans to introduce elements such as 3D maps around the world and content that can be experienced by multiple users by fixing them at specific locations in the world by strengthening The Lightship AR development kit and cloud in the future. We are currently accepting private beta sign-ups on our developer page. Related information can be found here.