Do They All Look the Same? Deciphering Chinese, Japanese and Koreans by Fine-Grained Deep Learning
This work addresses the challenge of distinguishing between East Asian nationalities for applications in tourism, e-commerce, and security, but it is incremental as it builds on existing neural network methods and datasets.
The paper tackled the problem of classifying Chinese, Japanese, and Korean faces using fine-grained deep learning, achieving an accuracy of 75.03% compared to a chance level of 33.33% and human accuracy of 38.89%, and identified distinctive facial attributes like bangs, smiling, and bushy eyebrows for each group.
We study to what extend Chinese, Japanese and Korean faces can be classified and which facial attributes offer the most important cues. First, we propose a novel way of obtaining large numbers of facial images with nationality labels. Then we train state-of-the-art neural networks with these labeled images. We are able to achieve an accuracy of 75.03% in the classification task, with chances being 33.33% and human accuracy 38.89% . Further, we train multiple facial attribute classifiers to identify the most distinctive features for each group. We find that Chinese, Japanese and Koreans do exhibit substantial differences in certain attributes, such as bangs, smiling, and bushy eyebrows. Along the way, we uncover several gender-related cross-country patterns as well. Our work, which complements existing APIs such as Microsoft Cognitive Services and Face++, could find potential applications in tourism, e-commerce, social media marketing, criminal justice and even counter-terrorism.