PTeacher: a Computer-Aided Personalized Pronunciation Training System with Exaggerated Audio-Visual Corrective Feedback

(CHI’21) Yaohua Bu¹, Tianyi Ma¹, Weijun Li, Hang Zhou, Jia Jia*, Shengqi Chen, Kaiyuan Xu, Dachuan Shi, Haozhe Wu, Zhihan Yang, Kun Li, Zhiyong Wu, Yuanchun Shi, Xiaobo Lu, Ziwei Liu
Second language (L2) English learners often find it difficult to improve their pronunciations due to the lack of expressive and personalized corrective feedback.We present a Computer-Aided Pronunciation Training system that provides personalized exaggerated audio-visual corrective feedback for mispronunciations to realize the appropriate degrees of audio and visual exaggeration when it comes to individual learners. Therefore, three critical metrics are proposed for both 100 learners and 22 teachers to help us identify the appropriate degrees of exaggeration. User studies demonstrate that our system rectify mispronunciations in a more discriminative, understandable and perceptible manner.
[Video][Paper]