Playing the piano on paper?

MagicPAPER is a touch interaction system developed by a research team at Chengdu University of Information Technology in China using paper and projector projection images.

Magic Paper is a system consisting of Microsoft’s Kinect, kraft paper, webcam, projector, and software that recognizes gestures and voices. It is a structure in which the movement of the hand drawn on the paper is analyzed by using a webcam, and based on this information, the projector projects an appropriate image according to the movement.

For example, you can project a white flashing image of the projector on the touch of the Pinao keyboard drawn on paper and the area where your finger touches, and simultaneously play the touched key notes, giving you the impression that a player is playing a real piano.

In the video, you can see that water accumulates in a bottle drawn on paper according to the movement of your hand, or an illustration is projected onto a block on paper, and various interactions take place according to the work performed on the paper. Related information can be found here .