Can audio-visual speech recognition outperform acoustically enhanced speech recognition in automotive environment?
Navarathna, Rajitha, Kleinschmidt, Tristan, Dean, David B., Sridharan, Sridha, & Lucey, Patrick J. (2011) Can audio-visual speech recognition outperform acoustically enhanced speech recognition in automotive environment? In Interspeech 2011, 27-31 August 2011, Firenze Fiera, Florence.
The use of visual features in the form of lip movements to improve the performance of acoustic speech recognition has been shown to work well, particularly in noisy acoustic conditions. However, whether this technique can outperform speech recognition incorporating well-known acoustic enhancement techniques, such as spectral subtraction, or multi-channel beamforming is not known. This is an important question to be answered especially in an automotive environment, for the design of an efficient human-vehicle computer interface. We perform a variety of speech recognition experiments on a challenging automotive speech dataset and results show that synchronous HMM-based audio-visual fusion can outperform traditional single as well as multi-channel acoustic speech enhancement techniques. We also show that further improvement in recognition performance can be obtained by fusing speech-enhanced audio with the visual modality, demonstrating the complementary nature of the two robust speech recognition approaches.
Impact and interest:
Citation counts are sourced monthly from and citation databases.
Citations counts from theindexing service can be viewed at the linked Google Scholar™ search.
|Item Type:||Conference Paper|
|Keywords:||Speech enhancement, robust speech recognition, audio-visual automatic speech recognition, synchronous HMM|
|Subjects:||Australian and New Zealand Standard Research Classification > ENGINEERING (090000) > ELECTRICAL AND ELECTRONIC ENGINEERING (090600) > Signal Processing (090609)|
|Divisions:||Past > QUT Faculties & Divisions > Faculty of Built Environment and Engineering
Past > Institutes > Information Security Institute
Past > Schools > School of Engineering Systems
|Copyright Owner:||Copyright 2011 [please consult the author]|
|Deposited On:||08 Sep 2011 22:29|
|Last Modified:||18 Oct 2011 11:05|
Repository Staff Only: item control page