Deepfake, which synthesizes virtual images or videos using AI technology, has developed to a stage where it is difficult to identify the authenticity with the human eye, and since it can easily create a face of a person that does not exist, it is also a big problem in social media. A research team at New York State University’s Buffalo Campus has developed a technology that detects such deep fake facial photos with light reflections on the pupils.
At first glance, the face of a real person and the face of a virtual person created by the hostile generation network GAN are all visible only as real face photos. However, when the eye is enlarged, the shape of the light reflected from the cornea is almost the same from side to side, while the deep fake is scattered from side to side.
Since both eyes can see the same thing, the real face usually matches even when viewed through the left and right eyes. However, in many cases, the GAN cannot accurately reproduce the similarity. Therefore, it is explained that by mapping the eye position and analyzing the light shining on both eyes, deepfakes can be accurately identified if the face photo is real.
When analyzed with a discrimination tool developed by the research team, the IoU score for real face photos was evaluated as 0.5824 to 0.8406, but for face photos synthesized with deep fakes, it was rated 0.2429 to 0.3512. This method of discrimination does not work when a picture is taken with only one eye, and has a disadvantage that accuracy is greatly reduced if the subject is not looking at the camera. You can also manually correct the pupil light to make the decision difficult.
The research team explains that although sophisticated deep fakes are difficult at this stage, many flimsy deep fake photos can be seen. He said he plans to do research to further enhance the effectiveness of this method in the future. Related information can be found here.