This article presents a segmental vocoder driven by ultrasound and optical images
(standard CCD camera) of the tongue and lips for a “silent speech interface” application,
usable either by a laryngectomized patient or for silent communication. The system is built
around an audio–visual dictionary which associates visual to acoustic observations for each
phonetic class. Visual features are extracted from ultrasound images of the tongue and from
video images of the lips using a PCA-based image coding technique. Visual observations of …