Print from discussion forum

Topic: A Question For The Experts: Multimodal Interfaces
Username: Turin_airport
Posted 2007-01-29 11:40:26 and read 2243 times.

I would like to have your opinion about the use of a "multimodal" interface in the cockpit during flight or on the ground.

I try to make myself clear: a "multimodal" device is a system that can be activated and controlled both using traditional buttons or by the user's voice: that means that the user can give voice commands, or press buttons to interact with the system and receive spoken and/or visual/acoustic feedback.

An example (purely hypothetical) could be the belt signal: nowadays it's a button with a light to indicate if it's on or off. Its "multimodal" version could be also activated (or deactivated) giving a voice command (for example: "sit belt on"/"sit belt off"), and receive a vocal feedback by the system (along with the visual one).

My question is: in your opinion this interaction modality could be usefull in airplanes to manage secondary devices (for obvious reasons not the main controls) and which devices or controls would benefit by the combined use of the voice and traditional buttons?

Thanks a lot for your replies.


P.s.: such devices are currently been implemented in the automotive field. Few weeks ago Microsoft announced they are working with Ford to sell things like these:

Topic: RE: A Question For The Experts: Multimodal Interfaces
Username: Airfoilsguy
Posted 2007-01-29 13:22:37 and read 2233 times.

My dad has a Mercedes S550 that has this. It does not work so well (at least on the car). For example, he was showing me the car and said something like "change radio 107" and the sunroof opened. I almost died laughing and he was not pleased. I don't think the technology is at the point where they can put it into airplanes. Would be great once all the bugs are worked out though, brings back memories of Firefox.

Topic: RE: A Question For The Experts: Multimodal Interfaces
Username: BAe146QT
Posted 2007-01-29 13:30:18 and read 2232 times.

"Set tranponder to... Oh, hi Jack! How are you?"

This should really be designed by a British company. We do farce better than anyone.

Topic: RE: A Question For The Experts: Multimodal Interfaces
Username: Turin_airport
Posted 2007-01-29 17:06:24 and read 2204 times.

Like every other technology, a voice interface works as well as the effort you put in designing and making it.

The current systems can reach a reliability of more then 97% of correct catches. All depends by how many memory you have available and by how easy to use is your interface.
So, technically speaking, I think it's not a big deal, the question is: is it useful? Can this kind of interface help the pilot (or the f/o)?

Many secondary commands come into my mind: in theory, using the voice to activate/deactivate a secondary command could let the pilot to be free to concentrate on the most important actions. But, is it true? And which commands could be the candidate?



Topic: RE: A Question For The Experts: Multimodal Interfaces
Username: Brenintw
Posted 2007-01-30 01:51:26 and read 2148 times.

My experience of voice-activated software has been woeful. Personally, I wouldn't trust it in an aircraft until a lot more work has been done on it. VA software is very accent-sensitive -- and it requires a huge amount of work to get the software accustomed to *your* voice.

I remember working on a version of IBM's "ViaVoice" software that we were evaluating for work. It took about two hours to set up the software to recognize my voice (I'm not American, so it really did struggle with my accent) and even after doing all that, it has a success-rate something less than 30%. More recently, Windows XP Tablet PC Edition had similar problems, although it did have a significantly higher success rate.

The biggest problem with VA in an aircraft would be the sheer amount of storage and processing power that would be required to compare voice-prints. The software would need to build up a database of voice-print references for every single pilot that ever flies that particular aircraft and then when the pilot issues a command, cross-reference the command against all the databases to match the voice-prints to guess at the correct command. In a private aircraft, or aircraft that is flown by three or four pilots only, that would be workable, but not a mainline aircraft for a major carrier (how many B744 pilots does BA have?)

Topic: RE: A Question For The Experts: Multimodal Interfaces
Username: Turin_airport
Posted 2007-01-30 11:34:45 and read 2121 times.

Brenintw,, no, the technology used nowadays is called "speaker independent". That means that the recognition engine is capable of understanding commands, no matter the accent or gender of the speaker (the only thing to pay attention to is speaking the same language of the system).

So, given that the technology is not the problem, I would like to know if pilots could find the use of voice (or better, multimodal) interfaces helpful when they are in the cockpit, and which actions could be made using the voice (for example: selecting a new map on the screen, or interrogate the system about flight parameters, or else).

That's the way the multimodal interfaces are used in the automotive field these days: they help the driver to concentrate on the driving, without having to search for buttons to manage secondary devices, like telephones, navigators, mp3 player, and so on.


The messages in this discussion express the views of the author of the message, not necessarily the views of or any entity associated with

Copyright © Lundgren Aerospace. All rights reserved.