Recognising audio-visual speech in vehicles using the AVICAR database

Abstract

Interacting with technology within a vehicle environment
using a voice interface can greatly reduce the effects of driver distraction. Most current approaches to this problem only utilise the audio signal, making them susceptible to acoustic noise. An obvious approach to circumvent this is to use the visual modality in addition. However, capturing, storing and distributing audio-visual data in a vehicle environment is very costly and difficult. One current dataset available for such research is the AVICAR [1] database. Unfortunately this database is largely
unusable due to timing mismatch between the two streams and
in addition, no protocol is available. We have overcome this
problem by re-synchronising the streams on the phone-number
portion of the dataset and established a protocol for further research.

This paper presents the first audio-visual results on this
dataset for speaker-independent speech recognition. We hope
this will serve as a catalyst for future research in this area.