Cognitive Science in the Cockpit

The pilots of modern aircraft have lots of different things to do. They have to follow numerous regulations, consult with charts, and communicate with themselves, their company, and air traffic control. And then, during the major portion of the flight, they have to sit in the cockpit with very little to do, but keep alert for unexpected problems. If the trip is long, say across an ocean, there can be hours of boredom.

Where are the cognitive aids for these tasks? Even the comfort of the flight crew is ignored. Only recently have decent places to hold coffee cups emerged. In older planes the flight engineer has a small desk for writing and for holding manuals, but the pilots don't. In modern planes there are still no places for the pilots to put their charts, their maps, or in some planes, their coffee cups. Where can the crew stretch their legs or do the equivalent of putting the feet up on the desk? And when it is mealtime, how does one eat without risking spilling food and liquids over the cockpit? The lighting and design of the panels seem like an afterthought, so much so that a standard item of equipment for a flight crew is a flashlight. If comfort is ignored, think how badly mental functioning must be treated.

The same is true all over, I might add. I have seen power plants where the operators have to stand on the instruments in order to change burned-out light bulbs, plants where there are no facilities for eating snacks and drinking coffee. It's as if the designers assumed that no equipment would ever burn out, that the workers would go for an entire eight-hour shift without nourishment.

Intuition and anecdote determine the training methods and the development of instrumentation, procedures and regulations. There has been almost no systematic attempt to provide support for cognitive activities. The crew has to fend for themselves, and because they sometimes recognize their own limitations, many have informally developed routines and "rules of thumb" to help them cope. Consider that many flight crews carry as standard equipment such wonderful cognitive tools as a role of tape and an empty coffee cup to be used as reminders on the instruments and controls. The empty coffee cup is actually quite effective when placed upside down over the throttle or flap handles to remind the pilots that some special condition applies to future use of these controls.

Why Is an Empty Coffee Cup Such a Powerful Cognitive Aid?

One of the most widely studied areas within cognitive science is that of human memory and attention. Although the final scientific theories have yet to be developed, we do know a lot. There is a considerable body of well-understood phenomena and several approximate theories that can be used to good effect in design. Among the simple lessons are that:

• We can only keep a very limited number of things consciously in mind at any one time in what is called "Working Memory." How few? Perhaps only five.



• Conscious attention is limited, so much so that it is best to think of a person as being able to focus on only one task at a time. Disruptions of attention, especially those caused by interrupting activities, lead to problems: people forget what was in working memory prior to the interruption and the interruption interferes with performance of the task they were trying to focus on.



• Internal information, "knowledge in the head," is subject to the limits posed by memory and attention. External information, "knowledge in the world," plays important roles in reminding people of the current state of things and of the tasks left to be done. Good design practice, therefore, will provide external aids to memory.

In general, people are not very accurate at tasks that require great precision and accuracy or precise memorization. People are very good at perceptual tasks, tasks that involve finding similarities (analogies) between one situation and another, and novel or unexpected tasks that involve creative problem solving. Unfortunately, more and more of the tasks in the cockpit force people to do just those tasks they are bad at and detract from the ability to do the things they are so good at.

How does the flight crew guard against problems? There are surprisingly few aids. Most of the aids in the cockpit are casual, informal, or invented by the crew in response to their own experiences with error. The most common (and effective) cognitive aids in the cockpit are:

"Speed bugs," little metal or plastic tabs that the pilots can move around the outside of the airspeed indicator to help them remember critical settings;

Crew-provided devices: written notes, coffee cups, and tape;

Checklists.

Speed bugs. Speed bugs are plastic or metal tabs that can be moved over the airspeed indicator to mark critical settings. These are very valuable cognitive aids, for they transform the task performed by the pilot from memorization of critical air speeds to perceptual analysis. The pilots only have to glance at the airspeed and instead of doing a numerical comparison of the airspeed value with a figure in memory, they simply look to see whether the speed indicator is above or below the bug position. The speed bug is an excellent example of a cockpit aid.

The speed bug is an example of something that started out as an informal aid. Some pilots used to carry grease pencils or tape and make marks on the dials. Today, the speed bugs are built into the equipment and setting them is part of standard procedures. Unfortunately, instrument designers have now gotten so carried away by the device that what used to be a single, easy to use tool has now been transformed into as many as five or more bugs set all around the dial. As a result, what was once a memory aid has now become a memory burden. I foresee speed bug errors as pilots confuse one bug with another. And, again, because of the lack of system knowledge, newer computer-displayed airspeed indicators sometimes neglect to include speed bugs or other memory aids for critical airspeed settings, sending us back to the dark ages of memory overload.

Crew-provided devices. Pilots and crew recognize their own memory deficiencies, especially when subject to interruptions. As a result, they use makeshift reminders in the cockpit. In particular, they rely heavily on physical marks. You know, want to remember something, tie a knot around your finger. Want to remember to take your briefcase, prop it against the door so you stumble over it when you go out. Want to remember to turn off the Air Conditioning Units before lowering the flaps, place an empty coffee cup over the flap handle. Crude, but effective.

Pilots need numerous reminders as they fly. They have to remember the flight number, radio frequencies, speed and altitude clearings. They need to know and remember the weather conditions. They may have special procedures to follow, or Air Traffic Control may have given them some special information.

But why hasn't this need been recognized? The need for mental, cognitive assistance should be recognized during the design of the cockpit. Why don't we build in devices to help the crew? Instead, we force them to improvise, to tape notes here and there, or even to wedge pieces of paper at the desired locations, all to act as memory aids. The crew needs external information, knowledge in the world, to aid them in their tasks. Surely we can develop better aids than empty coffee cups?

Cognitive aids. The need for memory aids applies to a wide range of human activities, in all professions and most human activities. Just think about your own activities. How many notes do you write to yourself? Why have Post-It™ notes become so popular? How many times have you forgotten something because you weren't reminded at the critical time? In everyday life, these issues are seldom of great importance, but in many industrial settings, they can be critical. There is a great need for cognitive aids in many aspects of life, aids that are designed with knowledge and understanding of human psychology.

Perhaps the one place where these problems are officially recognized is in the checklist, but even this appears to be an anomaly, designed more for the convenience of the training staff than for the needs of the crew.

Checklists

Chapter Note: My analyses of the checklists were done jointly with my colleague, Edwin Hutchins. Some of the information and analyses come from the excellent review by Degani & Wiener (1990).

One form of cognitive aid is in widespread existence: the checklist: a list of tasks to be performed. Personally, the checklist is an admission of failure, an attempt to correct for people's errors after the fact rather than to design systems that minimize error in the first place.

How are checklists used? In many different ways. In American commercial aviation, the "normal" checklists used in the cockpit are designed solely to serve as checks of actions already done. That is, the cockpit crew first sets up the cockpit for the flight, then uses the checklist to confirm that they did everything properly. In other situations (e.g., the "abnormal" checklists used in aviation when there are a problems) and in other industries, checklists are often used as reminders, guiding and suggesting actions to be performed. In these cases, they are read just before or as the action is done.

The very fact that checklists exist is admission that not all human behavior is perfect, that errors occur, and so, for safety and thoroughness, some items need to be especially "checked" to ensure that they are done.

Aviation checklists serve multiple functions:

1. As checks: to make sure that everything that was supposed to have been done was in fact done. This is actually the primary function of most aviation checklists. This is what the term "check list" really means.



2. As "triggers": to remind what needs to be done. This is how the "abnormal checklists" are used in aviation: when problems arise, the crew pull out the appropriate checklist for the problem and then go through it, doing each action as it is triggered by the list.



3. For Crew Communication. In most airlines, the entire flight crew takes part in going through the checklist: one person reads the items, one by one, while the others do the specified operations or check the specified state. This procedure ensures that the entire crew knows the state of the aircraft, especially anything unusual. This role of the checklist is often overlooked, but it may be one of its most important functions.



4. To Satisfy the Legal Department. Some items on the checklist are really not needed for anything except to guard against lawsuits. After an accident, during the court case, the legal experts want to make sure that the cockpit voice recording will distinctly show the pilots doing certain actions, even if they are not critically needed for flight safety. However, the longer the checklist, the more chance that there will be error in using it. Safety demands as short a list as possible: legal concerns demand longer lists.

But why do we need checklists at all? Checklists are not only a sign of human fallibility, they are also a sign that the procedures or equipment design is inappropriate. There are ways to design things to minimize the chance for skipping critical actions. A properly designed system might not even require a checklist.

Alas, each new requirement to aid the crew seems to result in new tasks for them to do, new procedures to be followed. I can see it now: my goal to help the pilots through my analysis of the valuable memory aid provided by empty coffee cups is mis-interpreted so as to add yet one more item to the procedures and checklists pilots must do. I can imagine a new checklist item: "Coffee cup supply?" Proper response: "Filled."

Blaming the Person -- A Way to Avoid the Real Issues

One last point: the prevalence to blame incidents on human error. In the past few years, human error has become the dominant blame for industrial accident. Thus, in the period 1982-1986, the pilot was blamed in 75% of fatal accidents.

Chapter Note: From the United States National Transportation Safety Board’s Annual review of aircraft accident data. U.S. air carrier operations calendar year 1987. (NTSB, 1990b).

Human error. How horrible! What's the matter with those pilots, anyway? Clearly they aren't being trained right. Fire them. Or at least send them back for more training. Change the training. Add some more flight regulations. Change the law. Add some more items to the checklists. This is what I call the "blame and train" philosophy.

Whenever I see such a high percentage of accidents blamed on individuals, I get very suspicious. When I am told that more than half of the world's accidents -- home and industrial -- are blamed on the people involved, I get very, very suspicious indeed. One way of thinking about the issue is this. If people are only rarely thought to be the culprit for some problem or accident, then maybe there is some reason to think that, in the exceptional case, the person did do something wrong. But if people often seems to be at fault, especially different people over long periods of time, then the first place to look for the explanation is in the situation itself.

Look, suppose it really is something in people that gives rise to accidents: shouldn't any sensible designer learn about those things and design the system so as to be resistant to that behavior or better yet, to avoid those situations? Alas, most engineers and designers are not well educated about human psychology. The psychological, cultural, and social knowledge relevant to human behavior is not part of the normal design or engineering training and education. Moreover, many designers fall prey to the "one chance in a million" syndrome (remember Chapter 15).

Until designers take seriously the usability of their designs and realize that inappropriate design is responsible for many accidents and casualties, we will never minimize such incidents. Let me give an example.

In 1988, the Soviet Union's Phobos 1 satellite was lost on its way to Mars. Why? According to Science magazine, "not long after the launch, a ground controller omitted a single letter in a series of digital commands sent to the spacecraft. And by malignant bad luck, that omission caused the code to be mistranslated in such a way as to trigger the test sequence" (the test sequence was stored in the computer memory of the satellite, but was to be used only during checkout of the spacecraft while on the ground). Phobos went into a tumble from which it never recovered.

Chapter Note: The report came from an editorial in Science magazine, (Waldrop, 1989), and formed the basis of an opinion piece by me in the Communications of the ACM, the primary journal for American computer scientists (ACM stands for Association for Computing Machinery). I argued that computer scientists need to pay a lot more attention to the design of their programs, else they too will cause Phobos-like accidents (Norman, 1990).

The American journal Science wrote its report as if the incompetence of the human controller had caused the problem. Science interviewed Roald Kremnev, director of the Soviet Union's spacecraft manufacturing plant. Here is how Science reported the discussion: "what happened to the controller who made the error? Well, Kremnev told Science with a dour expression, he did not go to jail or to Siberia. In fact, it was he who eventually tracked down the error in the code. Nonetheless, said Kremnev, he was not able to participate in the later operation of Phobos." Both the reporter's question and the answer presuppose the notion of blame. Even though the operator tracked down the error, he was still punished (but at least not exiled). But what about the designers of the language and software or the methods they use? Not mentioned. The problem with this attitude is that it prevents us from learning from the incident, and allows the same problem to be repeated.

This is a typical reaction to the problem -- blame the controller for the error and "malignant bad luck" for the result. Why bad luck -- why not bad design? Wasn't the problem the design of the command language that allowed such a simple deviant event to have such serious consequences?

The crazy thing is that normal engineering design practices would never allow such a state of affairs to exist with the non-human part of the equipment. All electrical signals are noisy. That is, if the signal is supposed to be 1.00 volts, well, sometimes it will be .94, sometimes 1.18. In a really bad environment, where there are lots of radio transmitters sending energy all about and large electrical cables and motors turning on and off, there might be occasional "spikes" that make the 1 signal jump to 5 volts or down to zero or even less, perhaps -5 volts, for a few thousandths of a second. This noise can really wreck the operation of a computer. Fortunately, there are numerous ways to protect against these problems.

The spacecraft designers -- who work in exactly this kind of a noisy environment -- work hard to minimize the effects of noise bursts. If they didn't, a burst of noise could turn a signal like 1 volt, which usually would encode the digital code for a "one" into just its opposite meaning, the digital code for a "zero." This could cause untold error in the operation of the computer. But, fortunately, there are many techniques to avoid problems. Designers are expected to use error-detecting and correcting codes.

Suppose, just suppose, that some sort of electrical noise had corrupted the signal sent from the ground to the Phobos satellite, causing it to be destroyed. Who would be at fault then? It certainly wouldn't be the ground controllers. No, the official verdict would probably state that the system designers did not follow standard engineering practice. The next time around that design would be redone so as to protect against future occurrences. Well, the same lesson applies to all situations: there is no excuse for equipment and procedures that are so sensitive to human error, and certainly no excuse for those so badly designed that they lead humans to err. People err, just as equipment does. Worse, most equipment seems designed so as to lead a person to err: all those tiny little switches, neatly lined up in an array of identical looking switches and readouts. Computer codes that are long, meaningless strings of letters and digits. Is it any wonder that sometimes the wrong switch gets pushed, the wrong gauge gets read, or a critical character isn't typed? Why would anyone design a computer control language so that a single typing error could lead to catastrophe? And why is a procedure meant to be used only the ground, still possible to activate once in space?

The fault is the design that completely failed to take into account the needs of the people who had to use it. Don't punish the controllers, change the design philosophy.

As automation increasingly takes its place in industry it is often blamed for causing harm and increasing the chance of human error when failures occur. I propose that the problem is not the presence of automation, but rather its inappropriate design. The problem is that the operations under normal operating conditions are performed appropriately, but there is inadequate feedback and interaction with the humans who must control the overall conduct of the task. When the situations exceed the capabilities of the automatic equipment, then the inadequate feedback leads to difficulties for the human controllers.

The problem, I suggest, is that the automation is at an intermediate level of intelligence, powerful enough to take over control that used to be done by people, but not powerful enough to handle all unusual conditions. Moreover, its level of intelligence is insufficient to provide the continual, appropriate feedback that occurs naturally among human operators. This is the source of the current difficulties. To solve this problem, the automation should either be made less intelligent or more so, but the current level is quite inappropriate. Either the person should be in control all the time or in continual communication with the automatic tools. The intermediate state where the automatic equipment is in control, but with no external feedback, no communication, is probably the worst of all possibilities, especially when trouble strikes.

Complex systems involve a mixture of automatic and human control. Alas, there is too much tendency to let the automatic controls do whatever activities they are capable of performing, giving the leftovers to people. This is poor system design. It does not take into account the proper mix of activities, and it completely ignores the needs and talents of people. The price we pay for such disregard for the total system performance comes when things go wrong, when unexpected conditions arise or the machinery breaks down. The total reliability and safety of our systems could be improved if only we understood and treated people with the same respect and dignity that we give to electronic signals and to machines.

< previous page | next page >

 

FOOTNOTE

1. Copyright © 1992 by Donald A. Norman. All rights reserved. Originally published by Addison Wesley. Now out of print. [Return to Text]