Lip Reading for robust speech recognition on embedded devices

In this article a complete audio-visual speech recognition system suitable for embedded devices is presented. As visual feature extraction algorithms Active Shape Models (ASM) and Discrete Cosine transformation (DCT) have been investigated and discussed for an embedded implementation. The audio-visual information integration has also been designed by taking into account device limitations. It is well known that the use of visual cues improves the recognition results especially in scenarios with high level of acoustical noise. We wanted to compare the performance of Lip Reading and the conventional Noise Reduction systems in these degraded scenarios, as well as the combination of both kinds of solutions. Important improvements are obtained especially for non-stationary background noises like voice interference, car accelerations or indicators clicks. For this kind of noises Lip Reading outperforms the results obtained with conventional Noise Reduction technologies.