Topic Author
Posts: 273
Joined: Mon Oct 08, 2001 5:29 pm

A Question For The Experts: Multimodal Interfaces

Mon Jan 29, 2007 7:40 pm

I would like to have your opinion about the use of a "multimodal" interface in the cockpit during flight or on the ground.

I try to make myself clear: a "multimodal" device is a system that can be activated and controlled both using traditional buttons or by the user's voice: that means that the user can give voice commands, or press buttons to interact with the system and receive spoken and/or visual/acoustic feedback.

An example (purely hypothetical) could be the belt signal: nowadays it's a button with a light to indicate if it's on or off. Its "multimodal" version could be also activated (or deactivated) giving a voice command (for example: "sit belt on"/"sit belt off"), and receive a vocal feedback by the system (along with the visual one).

My question is: in your opinion this interaction modality could be usefull in airplanes to manage secondary devices (for obvious reasons not the main controls) and which devices or controls would benefit by the combined use of the voice and traditional buttons?

Thanks a lot for your replies.


P.s.: such devices are currently been implemented in the automotive field. Few weeks ago Microsoft announced they are working with Ford to sell things like these:

Posts: 3485
Joined: Sun Oct 09, 2005 7:28 am

RE: A Question For The Experts: Multimodal Interfaces

Mon Jan 29, 2007 9:22 pm

My dad has a Mercedes S550 that has this. It does not work so well (at least on the car). For example, he was showing me the car and said something like "change radio 107" and the sunroof opened. I almost died laughing and he was not pleased. I don't think the technology is at the point where they can put it into airplanes. Would be great once all the bugs are worked out though, brings back memories of Firefox.
It's not a near miss it's a near hit!!
Posts: 981
Joined: Sun Sep 24, 2006 4:58 am

RE: A Question For The Experts: Multimodal Interfaces

Mon Jan 29, 2007 9:30 pm

"Set tranponder to... Oh, hi Jack! How are you?"

This should really be designed by a British company. We do farce better than anyone.
Todos mis dominós son totalmente pegajosos
Topic Author
Posts: 273
Joined: Mon Oct 08, 2001 5:29 pm

RE: A Question For The Experts: Multimodal Interfaces

Tue Jan 30, 2007 1:06 am

Like every other technology, a voice interface works as well as the effort you put in designing and making it.

The current systems can reach a reliability of more then 97% of correct catches. All depends by how many memory you have available and by how easy to use is your interface.
So, technically speaking, I think it's not a big deal, the question is: is it useful? Can this kind of interface help the pilot (or the f/o)?

Many secondary commands come into my mind: in theory, using the voice to activate/deactivate a secondary command could let the pilot to be free to concentrate on the most important actions. But, is it true? And which commands could be the candidate?


User avatar
Posts: 1579
Joined: Sun Jul 02, 2006 5:31 pm

RE: A Question For The Experts: Multimodal Interfaces

Tue Jan 30, 2007 9:51 am

My experience of voice-activated software has been woeful. Personally, I wouldn't trust it in an aircraft until a lot more work has been done on it. VA software is very accent-sensitive -- and it requires a huge amount of work to get the software accustomed to *your* voice.

I remember working on a version of IBM's "ViaVoice" software that we were evaluating for work. It took about two hours to set up the software to recognize my voice (I'm not American, so it really did struggle with my accent) and even after doing all that, it has a success-rate something less than 30%. More recently, Windows XP Tablet PC Edition had similar problems, although it did have a significantly higher success rate.

The biggest problem with VA in an aircraft would be the sheer amount of storage and processing power that would be required to compare voice-prints. The software would need to build up a database of voice-print references for every single pilot that ever flies that particular aircraft and then when the pilot issues a command, cross-reference the command against all the databases to match the voice-prints to guess at the correct command. In a private aircraft, or aircraft that is flown by three or four pilots only, that would be workable, but not a mainline aircraft for a major carrier (how many B744 pilots does BA have?)
Topic Author
Posts: 273
Joined: Mon Oct 08, 2001 5:29 pm

RE: A Question For The Experts: Multimodal Interfaces

Tue Jan 30, 2007 7:34 pm

Brenintw,, no, the technology used nowadays is called "speaker independent". That means that the recognition engine is capable of understanding commands, no matter the accent or gender of the speaker (the only thing to pay attention to is speaking the same language of the system).

So, given that the technology is not the problem, I would like to know if pilots could find the use of voice (or better, multimodal) interfaces helpful when they are in the cockpit, and which actions could be made using the voice (for example: selecting a new map on the screen, or interrogate the system about flight parameters, or else).

That's the way the multimodal interfaces are used in the automotive field these days: they help the driver to concentrate on the driving, without having to search for buttons to manage secondary devices, like telephones, navigators, mp3 player, and so on.


Who is online

Users browsing this forum: No registered users and 14 guests

Popular Searches On Airliners.net

Top Photos of Last:   24 Hours  •  48 Hours  •  7 Days  •  30 Days  •  180 Days  •  365 Days  •  All Time

Military Aircraft Every type from fighters to helicopters from air forces around the globe

Classic Airliners Props and jets from the good old days

Flight Decks Views from inside the cockpit

Aircraft Cabins Passenger cabin shots showing seat arrangements as well as cargo aircraft interior

Cargo Aircraft Pictures of great freighter aircraft

Government Aircraft Aircraft flying government officials

Helicopters Our large helicopter section. Both military and civil versions

Blimps / Airships Everything from the Goodyear blimp to the Zeppelin

Night Photos Beautiful shots taken while the sun is below the horizon

Accidents Accident, incident and crash related photos

Air to Air Photos taken by airborne photographers of airborne aircraft

Special Paint Schemes Aircraft painted in beautiful and original liveries

Airport Overviews Airport overviews from the air or ground

Tails and Winglets Tail and Winglet closeups with beautiful airline logos