When you meet augmented reality … Soccer game on the table?

With the opening of the Russia World Cup, soccer frenzy is blowing all over the world. Unless you are a host country, most of you will have to watch a TV game with a 2D game. But what if you could use augmented reality technology to see a soccer match on the desk in three dimensions?

This technology was announced by Konstantinos Rematas of the University of Washington. He presented Soccer on your Tabletop ( http://grail.cs.washington.edu/projects/soccer/ ), a technology soccer game that can be rendered on the table by adding augmented reality technology to the football game. The system rearranges the 3D player data extracted from the soccer game scene into 3D space using CNN (Convolutional Neural Network), transforms the 2D game image into 3D, and freely watches the soccer game in various directions.

When I look at the table with the Hololens, Microsoft’s composite reality headset, the football game is unfolding in front of me. The user can freely watch the game from any angle of 360 degrees. The video itself was actually on YouTube. As I said before, I used CNN and game data to reconstruct 3D CG. This technique recognizes the line in the soccer field and calibrates the camera through it. Detect the players on the line and then track the player’s movements to distinguish the players and the lines. By tracking the movement of the athlete, you can estimate several movements and see which of the left and right legs are in front. Here, we use the player data in the soccer game to reconstruct the game itself. Transforming images through this process.

Of course, this technology is still in the development stage, so player movement is not accurate and resolution is not perfect. However, reconstructing stereoscopic images that can view images in all directions and angles can be a way of watching future enough to be interested. The source code for this technology is available in february.

The technology also used Microsoft’s holographic lens, but in fact, Microsoft had imagined it in 2016 when it unveiled a concept image of a futuristic sporting event using a holographic lens. It was a scene of watching the NFL game scene at home in a three-dimensional state with a holocled lens.

By putting the lens alone, you can feel the immersive feeling as if you are in the arena by using the wall or the whole room as a screen, away from the TV screen. If you move the viewpoint vertically and horizontally, you can watch 360 degrees in any direction as long as the image that moves on the wall moves according to your body movements. Like the technique described above, you can also use the table to display stadium information. This allows you to see information about individual players. Because the game information is reflected in real time, you can concentrate on the game without using a device such as a smart phone.

This technology is still in development, but it is enough to draw attention from the fact that it has entered the stage of development toward conceptualization.