Translated Abstract
For videos of oral presentations using visual materials such as photographs and diagrams, it is known that the viewers' comprehension is improved if the presenters appear in the frame with gestures. However, the necessity of gestures should depend on the linguistic expressions used with those gestures, and it is not clear whether the viewers actually pay attention to the gestures regardless of their necessity. This paper describes an early result of our gaze analysis for the viewers watching videos in which an avatar describes a route with linguistic expressions and gestures by showing a map.
References
- [1] Wang, J., :Mind wandering in videos that integrate instructor’s visuals: An eye tracking study, Innovations in Education and Teaching International (2023).
- [2] 小島隆次: 授業動画におけるバーチャルアバターがもたらす学習者への効果 ― バーチャルアバターの有無と動きに注目して, 日本認知科学会第38回大会論文集, pp.560-563 (2021).
- [3] Schneider, B., Sung, G., Hassan, J., Bhinder, H.: What matters most for learning from online videos, seeing the instructor’s face or gaze? Impact on instructor-student synchrony, Proceedings of the International Conference on Computer-Supported Collaborative Learning, pp.3-10 (2022).
- [4] Pi, Z., Zhang, Y., Yu, Q., Zhang, Y., Yang, J., Zhao, Q.: Neural oscillations and learning performance vary with an instructor’s gestures and visual materials in video lectures,” British Journal of Educational Technology, Vol.53, pp.93-113 (2022).
- [5] Özer, D., Göksun, T.: Visual-spatial and verbal abilities differentially affect processing of gestural vs. spoken expressions, Language, Cognition and Neuroscience, Vol.35, No.7, pp.896- 914 (2020).
- [6] Özer, D., Karadöller , D. Z., Özyürek, A., Göksun, T.: Gestures Cued by Demonstratives in Speech Guide Listeners’ Visual Attention During Spatial Language Comprehension, Journal of Experimental Psychology: General, Vol.152, No.9, pp.2623- 2635 (2023).
- [7] Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., Hays, J.: WebGazer: Scalable Webcam Eye Tracking Using User Interactions, Proceedings of International Joint Conference on Artificial Intelligence, pp.3839-3845 (2016).
- [8] Cao, Z., Hidalgo Martinez, G., Simon, T., Wei, S.: OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.172-186 (2021).
