Movie to Manga… Fully automatic generation system

Film comics is a film comic that cuts a large amount of photos from video works such as movies and animations, arranges them like a cartoon cut, and adds dialogue and sound effects. The research team of Dalian Academy of Science and Technology in China developed a system that makes such a movie fully automatic as an editor, cartoon.

The system developed by the research team extracts key frames from the video, converts them into comic-like images, and divides the frames across multiple pages, increasing the image according to the time series. The dialogue for each character is written as a balloon dialogue extracted from the subtitle file included in the voice or video. In addition, the size of the dialogue character or the shape of the balloon is analyzed by the voice, and the system evaluates the character emotion and changes accordingly.

In addition, it is possible to determine who is speaking by capturing the actor’s mouth on the screen by a facial recognition system.

According to the research team, in some cases where the accuracy of the video key frame selection is still inferior, the same key frame is selected, so the generated cartoon may be verbose. However, the research team revealed that the experiment shows that compared to the previous system, it is possible to synthesize a richer and more attractive manga. Related information can be found here.