An AI-Based Feedback Visualisation System for Speech Training

Wynn, Adam T.; Wang, Jingyun; Umezawa, Kaoru; Cristea, Alexandra I.

doi:10.1007/978-3-031-11647-6_104

An AI-Based Feedback Visualisation System for Speech Training

Wynn, Adam T.; Wang, Jingyun; Umezawa, Kaoru; Cristea, Alexandra I.

Authors

Adam Wynn adam.t.wynn@durham.ac.uk
PGR Student Doctor of Philosophy

Dr Jingyun Wang jingyun.wang@durham.ac.uk
Assistant Professor

Dr Kaoru Umezawa kaoru.umezawa@durham.ac.uk
Assistant Professor

Professor Alexandra Cristea alexandra.i.cristea@durham.ac.uk
Professor

Contributors

Maria Mercedes Rodrigo
Editor

Noburu Matsuda
Editor

Professor Alexandra Cristea alexandra.i.cristea@durham.ac.uk
Editor

Vania Dimitrova
Editor

Abstract

This paper proposes providing automatic feedback to support public speech training. For the first time, speech feedback is provided on a visual dashboard including not only the transcription and pitch information, but also emotion information. A method is proposed to perform emotion classification using state-of-the-art convolutional neural networks (CNNs). Moreover, this approach can be used for speech analysis purposes. A case study exploring pitch in Japanese speech is presented in this paper.

Citation

Wynn, A. T., Wang, J., Umezawa, K., & Cristea, A. I. (2022). An AI-Based Feedback Visualisation System for Speech Training. In M. Mercedes Rodrigo, N. Matsuda, A. I. Cristea, & V. Dimitrova (Eds.), Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium (510-514). Springer Verlag. https://doi.org/10.1007/978-3-031-11647-6_104

Online Publication Date	Jul 26, 2022
Publication Date	2022
Deposit Date	Aug 8, 2022
Publicly Available Date	Jul 27, 2023
Publisher	Springer Verlag
Pages	510-514
Series Title	Lecture Notes in Computer Science
Series Number	13356
Book Title	Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium
ISBN	978-3-031-11646-9
DOI	https://doi.org/10.1007/978-3-031-11647-6_104
Public URL	https://durham-repository.worktribe.com/output/1644590