Home | deutsch  | Legals | Sitemap | KIT

Silent Speech Interfaces

Das Cognitive Systems Laboratory im Institut für Anthropomatik des Karlsruher Instituts für Technologie (KIT) arbeitet an Silent Speech Interfaces, die es Menschen erlauben, voellig lautlos zu sprechen und dennoch vom Gesprächspartner verstanden zu werden. Im März 2010 haben wir unseren Prototypen zum ersten Mal auf der CeBIT, der weltgrößten IT-Messe, vorgestellt. Eine Vorabdemonstration gaben wir bereits auf der CeBIT VISION-Presseschau am 24. November 2009.

The Cognitive Systems Laboratory at the Institute for Anthropomatics of Karlsruhe Institute of Technology (KIT) develops innovative Silent Speech Interfaces, which allow humans to communicate with each other by speaking silently. In march 2010, we demonstrated our prototype on the CeBIT, the world's largest IT fair, for the first time. We also presented the technology on the CeBIT Vision press show on November 24, 2009.

Technology

The technology is based on Electromyography, i.e. the capturing and recording of electrical potentials that arise from muscle activity. Speech is produced by the contraction of muscles that move our articulatory apparatus. The electrical potentials which are generated by this muscular activity are captured by surface electrodes attached to the skin. The analysis and processing of these signals by suitable pattern matching algorithms allow to reconstruct the corresponding movement of the articulatory muscles and to deduct what has been said. The recognized speech is output as text or synthesized as an acoustic signal. Since electromyography records the muscle activity rather than acoustic signals, speech can be recognized even if it is uttered silently, without any sound production.

Applications

Our group has been working on this technology since 2004; the prototype will be presented at CeBIT 2010 for the first time, showing the following applications:

  1. Silent Telephony: Silent speech recognition allows for silent communication without disturbing any bystanders.
  2. Transmitting Confidential Information: The system allows for seamless switches between silent and audibly spoken speech and thus enables to safely and securely transmit confidential information such as passwords and PINs.
  3. Robust communication in adverse environments: Since electromyography relies on signals captured directly at the human body, the signal is not corrupted by noisy and adverse conditions.
  4. Speaking in a foreign tongue: By feeding the output of silent speech recognition into a component that translates from one language to another, native speakers can silently utter a sentence in their language, and the receivers hear the translated sentence in their language. It appears as if the native speaker produced speech in a foreign language.
  5. Help for disabled people: Our technology may also help people who have lost their voice due to accident or illness.

Press echo and press information 

Maybe you have heard or read about the Cognitive Systems Lab or our prototype?

Our press echo may be found here.

Also, you may look at our Demo Videos.

Publications 

The following publications give an overview on our scientific work in the field of EMG-based speech recognition. For a complete list, please have a look at the publications list of the Cognitive Systems Lab

  • Michael Wand and Tanja Schultz
    Session-Independent EMG-based Speech Recognition
    International Conference on Bio-inspired Systems and Signal Processing, Biosignals 2011, Rome, Italy
    Download
    A good introduction into our current work.

  • Tanja Schultz and Michael Wand
    Modeling Coarticulation in EMG-based Continuous Speech Recognition
    Speech Communication 52 (2010), pages 341-353
    Download
    Extensive article about describing articulatory movements. 

  • EMG data corpus
    a downloadable trial about the EMG-UKA data corpus of electromyographic and acoustic recordings of speech.
    Download

 

Contact

You may direct further inquiries to the following persons:

Prof. Dr.-Ing. Tanja Schultz
Cognitive Systems Lab (Director), Institute for Anthropomatics
Karlsruhe Institute of Technology
Adenauerring 4, 76131 Karlsruhe, Germany
Tel. +49 721/608-46300
tanja.schultz∂kit.edu

Dipl. Inform. Matthias Janke
Cognitive Systems Lab, Institut für Anthropomatik und Robotik
Karlsruhe Institute of Technology
Adenauerring 4, 76131 Karlsruhe, Germany
Tel. +49 721/608-45059
matthias.janke∂kit edu