<!--intro-->Imagine being able to land a jumbo jet without ever taking control of the stick. NASA scientists recently demonstrated the ability to control a 757 passenger jet simulation, using only human muscle-nerve signals linked to a computer.<!--/intro-->
Scientists outfitted the pilot with an armband implanted with eight electrodes. The sensors read muscle nerve signals as the pilot made the gestures needed to land a computer-generated aircraft at San Francisco International Airport in California. The pilot also demonstrated the ability to land a damaged aircraft during emergency landing drills. The work was reported in the October 2000 proceedings of the World Automation Congress.
"This is a fundamentally new way to communicate with machines -- another way to talk with our mechanical world," said the paper's principal author, Dr. Charles Jorgensen, head of the neuroengineering laboratory at NASA's Ames Research Center, Moffett Field, CA. The other authors are fellow Ames researchers Dr. Kevin Wheeler and Dr. Slawomir Stepniewski. "This new technology is significant in that neuroelectric control of computers can replace computer keyboards, mice and joysticks for some uses," Jorgensen added.
"In the experiment, a pilot closes his fist in empty air, makes movements and creates nerve signals that are captured by a dry electrode array on his arm," said Jorgensen. "The nerve signals are analyzed and then routed through a computer, allowing the pilot to control the simulated airplane." The pilot sees the aircraft and control panel projected on a large, dome-shaped screen while flying the aircraft.
Engineers made the first prototype armband from exercise tights, and used metallic dress-buttons as dry electrodes. "An advantage of using neuroelectric machine control is that human nerve signals can be linked directly with devices without the aid of joysticks or mice, thereby providing rapid, intuitive control," Jorgensen added. "This technology also is useful for astronauts in spacesuits who need to control tools in space."
Neuroelectric control uses "neural net" software that "learns" patterns that can slowly change and evolve with time, as well as combining many patterns together to generate a response.
Nerve signal patterns, each of which is potentially as unique as a fingerprint, are a perfect application for neural net software. A particular nerve-signal pattern tells muscles to move in a certain way. A computer can match each unique nerve-signal pattern with a particular gesture, such as making a fist or pointing. Scientists designed software that can adjust for each pilot's nerve patterns, which can be affected by caffeine use, biorhythms, performance stress and the amount of fat under the skin.
To demonstrate bioelectric muscle control of the simulated 757 airplane during emergencies, researchers combined this technology with two other NASA developments, the ability of the neural net software to learn to fly damaged airplanes, and propulsion-only landing of aircraft.
In about one-sixth of a second, a computer onboard a damaged aircraft can "relearn" to fly a plane, giving the pilot better control. Severe damage, such as partially destroyed wings, fuselage holes or sensor failures greatly alter how an airplane handles, and a pilot's controls may respond oddly or might not work at all, according to Jorgensen.
"When we combined the three technologies, the neuroelectrically wired pilot took the simulated aircraft into landing scenarios with a cascading series of accidents, first locking rudder controls and then progressing to full hydraulic failure," said Jorgensen. "For each case, successful landings were demonstrated for autopilot, damaged and propulsion-only control."
Scientists outfitted the pilot with an armband implanted with eight electrodes. The sensors read muscle nerve signals as the pilot made the gestures needed to land a computer-generated aircraft at San Francisco International Airport in California. The pilot also demonstrated the ability to land a damaged aircraft during emergency landing drills. The work was reported in the October 2000 proceedings of the World Automation Congress.
"This is a fundamentally new way to communicate with machines -- another way to talk with our mechanical world," said the paper's principal author, Dr. Charles Jorgensen, head of the neuroengineering laboratory at NASA's Ames Research Center, Moffett Field, CA. The other authors are fellow Ames researchers Dr. Kevin Wheeler and Dr. Slawomir Stepniewski. "This new technology is significant in that neuroelectric control of computers can replace computer keyboards, mice and joysticks for some uses," Jorgensen added.
"In the experiment, a pilot closes his fist in empty air, makes movements and creates nerve signals that are captured by a dry electrode array on his arm," said Jorgensen. "The nerve signals are analyzed and then routed through a computer, allowing the pilot to control the simulated airplane." The pilot sees the aircraft and control panel projected on a large, dome-shaped screen while flying the aircraft.
Engineers made the first prototype armband from exercise tights, and used metallic dress-buttons as dry electrodes. "An advantage of using neuroelectric machine control is that human nerve signals can be linked directly with devices without the aid of joysticks or mice, thereby providing rapid, intuitive control," Jorgensen added. "This technology also is useful for astronauts in spacesuits who need to control tools in space."
Neuroelectric control uses "neural net" software that "learns" patterns that can slowly change and evolve with time, as well as combining many patterns together to generate a response.
Nerve signal patterns, each of which is potentially as unique as a fingerprint, are a perfect application for neural net software. A particular nerve-signal pattern tells muscles to move in a certain way. A computer can match each unique nerve-signal pattern with a particular gesture, such as making a fist or pointing. Scientists designed software that can adjust for each pilot's nerve patterns, which can be affected by caffeine use, biorhythms, performance stress and the amount of fat under the skin.
To demonstrate bioelectric muscle control of the simulated 757 airplane during emergencies, researchers combined this technology with two other NASA developments, the ability of the neural net software to learn to fly damaged airplanes, and propulsion-only landing of aircraft.
In about one-sixth of a second, a computer onboard a damaged aircraft can "relearn" to fly a plane, giving the pilot better control. Severe damage, such as partially destroyed wings, fuselage holes or sensor failures greatly alter how an airplane handles, and a pilot's controls may respond oddly or might not work at all, according to Jorgensen.
"When we combined the three technologies, the neuroelectrically wired pilot took the simulated aircraft into landing scenarios with a cascading series of accidents, first locking rudder controls and then progressing to full hydraulic failure," said Jorgensen. "For each case, successful landings were demonstrated for autopilot, damaged and propulsion-only control."