VR controller that can recognize movement for each finger

The Sony Interactive Entertainment research team developed a next-generation virtual reality controller that can precisely track the movement of each finger and disclosed the results of its prototype research.

This virtual reality controller holds your hand in a virtual space by holding a clasp-like part. It can even detect bending of one finger one by one and immediately reflect it in the 3D image. It detects when you clench your fist and then open your finger. You can recognize which finger is bent and unfolded. Even hands of different sizes can recognize up to 11 gestures.

However, there are also unknown finger movements. For example, it is difficult to recognize movements such as attaching the index and middle fingers, and the ring and pinky fingers and widening only the middle and ring fingers. Likewise, movements such as making a loop by attaching a forefinger and thumb cannot be recognized.

The controller easily recognizes sweaty hands, hands with rings, watches, etc. It has 62 electrodes on its own and tracks finger movements on the camera in a system that determines finger movement from the capacitance it detects per electrode. It also uses an optical tracking system and a circuit neural network that learns finger movement corresponding to the capacitance applied to each electrode.

Of course, this controller is only in the stage of publishing the paper, and whether it will be put into practical use is not yet known. However, Sony acquired a license agreement for skin sensory feedback technology for virtual reality controllers in May of last year, and plans to install this technology in DualSense, a controller for PlayStation 5. It is also pointed out that the technology announced this time can be applied to such a controller. Related information can be found here .