What is automation?

Automation is the definition for when a machine performs a task that was normally performed by the human operator. It has some ironies. For example, when it works well, we trust it. However, sometimes it fails, and those failures can be catastrophic. Think of airplane crashes, of which the consequences are severe.

Often, these failures are not solely related to software or hardware components. Instead, the problem is often in human issues of attention, perception, and cognition in managing the automated system. The performance of most automation depends on the interaction between people with the technology.

Why do people automate?

There are four different categories of reasons for why designers develop machines to replace or aid human performance:

1. Impossible or hazardous. Sometimes processes are automated because it is impossible or dangerous for humans to perform the task. For example, think of teleoperation or robotic handling of hazardous material. Sometimes there are also special populations who have disabilities that lead them to be unable to carry out skills without assistance. Examples of these are automatic guidance systems for the quadriplegic or automatic readers for the visually impaired. Thus, automation often enables people to do what would otherwise be impossible.
2. Difficult or unpleasant. Sometimes the tasks are not impossible, but very difficult for humans. For example, humans can also calculate digits, but it is more effortful and error-prone compared to an automatic calculator.
3. Extend human capability. Sometimes automated functions may not replace, but simply aid humans in doing things. For example, the human working memory is vulnerable to forgetting. Then automated aids that supplement memory are useful. For example, automated telephone operators that directly print the desired phone number on a small display on your telephone. Automation is particularly useful in extending human’s multitasking capabilities: pilots report that autopilots can be useful in temporarily reliving them from duties of aircraft control when other task demands make their workload extremely high.
4. Technically possible. Sometimes, automated functions exist because they CAN exist (the technology is available and it is inexpensive). This does not always add value to the human user. For example, when we are calling a service, we often go through a menu which redirects us to the specific help desk. However, it would save the callers a lot more time if a speaker directly picks up the phone. But, for the company, the menu system is way cheaper than the actual person. According to the authors of the book, automation should focus on supporting system performance and humans’ tasks rather than being about technical sophistication.

What are the stages and levels of automation?

To explain what automation is, it can be useful to talk about the stages of human information processing that it replaces, and also the amount of cognitive or motor work that automation replaces (the level of automation). There are four stages of automation, with different levels in each stage:

1. Information acquisition, selection and filtering. Automation is a replacement for many cognitive processes of human selective attention. Examples are spell-checker systems in Word which redline misspelled words. Other, more aggressive examples of stage 1 automation are those that filter or delete information which is assumed to be irrelevant.
2.  Information integration. Automation serves as a replacement for many of the cognitive processes of perception and working memory, in order to provide the operator with a situation assessment, inference, diagnosis, or easy-to-interpret picture. Examples are automation at stage 2 are visual graphics that are configured in a way that makes perceptual data easier to integrate. At higher levels are automatic pattern recognizers, predictor displays, and diagnostic expert systems. A lot of intelligent warning systems that guide attention (stage 1) also include integration.
3. Action selection and choice. In stage 2, there are automated aids that diagnose a situation. These are different from those that recommend a particular course of action. In stage 3, there is an automated agent which assumes a certain set of values for the operator who is dependent on its advice. For example think of the airborne traffic alert and collision avoidance system (TCAS) which advises the pilot of a maneuvre to avoid colliding with another aircraft.
4. Control and action execution. Examples of control automation are autopilots in aircraft, cruise control in driving, and robots in industrial processing.

At stages 3 and 4, the level of automation is so high that it is of critical importance.

What are possible problems in automation?

As noted, there are shortcomings in automation. However, when talking about the shortcomings, one should never forget about the benefits of automation. For example, the ground proximity warning systems in aircraft has helped to save many lives by alerting pilots to possible crashes.

What is automation reliability?

Automation is reliable, when it does what the human operator wants it to do. However, for human interaction, perceived reliability is more important than actual reliability. There are four reasons for why automation may be perceived as unreliable:

1. It is indeed unreliable. This is the case when a component fails, or when the design has flaws. It is important to note that automated systems are often complex and consist of more components and are therefore more prone to errors in creating them.
2. There may be certain situations in which the automation is not designed to operate or in which it may not operate well. All automation are created to be used for a limited operation range. Then, using automation for other purposes may lead to lower reliability. For example, cruise control is used to maintain a constant speed level on a highway. It does not slow the car when going off a steep hill.
3. The human operator may incorrectly ‘set up’ the automation. For example, nurses sometimes make errors when they program systems that allow patients to administer periodic of painkillers. If they enter a wrong dose, the system will perform it anyhow.
4. Sometimes, the automated system does exactly what it intends to do, but it is so complex that the human operator does not fully understand it. This makes the automation seem like it is acting erroneously to the operator.

According to the authors, it is better to say ‘imperfect’ automation than ‘unreliable automation’, because automation is often used for tasks which are impossible to do perfectly (weather forecasting).

What about trust?

Trust is related to perceived reliability. We trust others, when we know that they do what is expected. The same goes for automated systems. Trust should thus be well calibrated: our trust in the agent should be in direct proportion to its reliability. Mistrust refers to when trust is not directly related to reliability: as reliability decreases, our trust should go down and we should be prepared to act ourselves and be receptive to sources of advice or information.

Studies have shown that human trust in automation is not fully well calibrated: sometimes there is distrust (too little trust) and sometimes there is overtrust (too much trust). As an example of distrust, think of circumstances in which people prefer manual control over automatic control, such as in the case of automation that enhances perception: people still want to see for themselves. There is also distrust of alarm systems with many false alarms. Distrust in automation can result from a failure to understand the system. The consequences of distrust are not always severe, but they may lead to inefficiency when this leads people to reject the good assistance that automation can offer.

What about overtrust and complacency?

Overtrust is sometimes called ‘complacency’.  This means that people trust the automation more than they should. This can have severe negative consequences. For example, if a airline pilot trusts his automation too much, this can result in a crash. Overtrust results from positive experiences: most often, automated systems do work well. Sometimes there are no failures at all. However, this does not mean that the automated system is perfect. When people overtrust the automated systems, they can stop monitoring it. This is a problem, when the system does fail!

There are three distinct implications of this:

1. Detection. When there is overtrust, the operator will be slower to detect a real failure.
2. Situation awareness. When people are active participants of the process, they are better aware of the dynamic state of processes, and they will be better in selecting and executing actions compared to when they are passive monitors. When people are thus distracted, or do not fully understand the system, this may lead them to be less likely to intervene correctly and appropriately.
3. Skill loss. When operators become passive monitors, this leads to ‘deskilling’, or gradual skill loss. This can have two consequences: the operator becomes less confident in his or her own performance, and becomes more likely to continue to use automation. Second, it may hinder appropriate actions when the system fails.

Another irony is that the circumstances in which automation devices fail, are the same circumstances that are most challenging to humans. This thus means that automation fails, when humans need it the most. However, as humans have learned to trust the automated systems, they might be unable to perform the task themselves.

What about workload and situation awareness?

One goal of automation is to reduce operator workload. For example, an automated device for lane keeping can help the driver to reduce driving workload. However, sometimes automation replaces workload in situations in which the workload was already very low and instead of workload, the loss of arousal is the problem. Sometimes, the reduced workload can also result in lower situation awareness: as automation level goes up, both workload and situation awareness go down. Sometimes automation reduces workload during already low-workload tasks and sometimes it increases workload during high-workload tasks. This is called clumsy automation: easy tasks become easier, and harder tasks become harder.

What about training and certification?

Automation can also lead to that complex tasks are perceived as being easy. This can then lead to reduction in trainings. For example, on ships, there was a lot of misunderstanding about the new radar and collision avoidance systems. This has contributed to accidents.

What about human cooperation?

In nonautomated systems, there are many circumstances in which communications are important. Sometimes this negotiation between humans is eliminated with automation, and this can be frustrating when humans try to interact with an uncaring, automated phone menu.

What about job satisfaction?

This book does not discuss the moral implications of replacing workers by automation. Many operators are highly satisfied and proud with their job, and when this person is then asked to remain in a potential position in case that the automation fails, could lead to negative and unpleasant situations.

What about function allocation between the person and automation?

Automation can be designed to avoid problems with operators. This could be done by systematic allocation of functions to the humans and to the automation, based on the capabilities of each. For example, one could allocate functions depending on whether the automation or human performs the function better. This begins with a task and function analysis. Functions are considered in terms of the demands that they place on the human and automation. This guides the decision to automate each function. As an example, think of a maritime navigitation which involves the position and velocity of surrounding ships using radar signals. This function involves complex operations. Then, automation is better at performing this. In contrast, the course selection involves judgement regarding how to interpret the rules of the road. Humans are better at exercising judgment, and then this task should be allocated to the human. In Table 2 of the book, you can see what things humans are better at compared to what automation is better at. I suggest you take a look at this, since this can be asked on the exam.

What about human-centered automation?

It seems better to think about how automation can support and complement humans, instead of limiting function allocation to one of the two. It is best that the automation design focuses on creating a human-automation partnership by incorporating the principles of human-centered automation. This could mean that the human has more authority over the automation, that a level of human involvement is chosen that leads to the best performance or that the worker’s satisfaction with the workplace is enhanced. There are six human-centered automation features that the authors believe will achieve the goal of maximum harmony between human, system, and automation:

1. Keeping the human informed. It is important for the operator to be informed about what the automation is doing and why. This can be done via displays. Thus, the pilot should be able to see the amount of thrust delivered by an engine as well as the amount of compensation that the autopilot might have to make to keep the plane flying straight.
2. Keeping the human trained. Automation often changes the task, and therefore operators should perform more abstract reasoning and judgment. Training for the automation-related demands is needed. Also, in case of failure, the operator’s skills should be as high as possible to avoid problems.
3. Keep the operator in the loop. This is one of the hardest challenges of human-centered automation. The question is: how can we keep operators in the control loop, so that awareness remains high? It seems that as long as the human maintains some involvement in decision making regarding whether to accept the automation suggestions or not, there are adequate levels of situation awareness. This is true even when workload was reduced.
4. Selecting appropriate stages and levels when automation is imperfect. Designers have to choose the stage and level of automation to incorporate into a system. It seems that the extent to which the automation is imperfect, the negative consequences of late stage imperfection are more harmful than early stage imperfection. Therefore, in implementing the recommendation for levels and stages in automation for high-risk decisions, it is important to realize the effect of time pressure. If a decision has to be made in a time-critical situation, later stages of automation can usually be done faster than by human operators.
5. Make the automation flexible and adaptive. The amount of automation needed for any task varies from person to person and within a person over time. A flexible automation system in which the level can vary is thus preferable over one that is fixed and rigid. Flexible automation means that there are different levels of automation: one driver may choose to use cruise control, the other may not. Adaptive automation is a bit different. In adaptive automation, the level of automation is based on particular characteristics of the environment, user, and task. For example, an adaptive automation system would be one in which the level of automation increases as the workload increases, or as the operator’s capacity decreases (fatigue). For example, when a system detects a high workload, the degree of automation can be increased.
6. Maintain a positive management philosophy. The management philosophy influences a worker’s acceptance and appreciation of automation. If they feel like automation is ‘imposed’ because it does the job better than that they do, they might have negative attitudes. However, if they see it as a complement on their performance, it will be accepted more.

What about supervisory control and automation-based complex systems?

Process control

Process control, such as during manufacturing of petro-chemicals, nuclear or conventional energy, the systems are so complex that there needs to be high levels of automation. Then the question is how to support supervisors in times of failures and fault management. This can be achieved using interfaces. These interfaces have two important features:

1. They are highly graphical. They use configural displays which represent the constraints on the system, in ways that these constraints can be easily perceived without heavy computations.
2. They allow the supervisor to think flexibly at different levels of abstraction, ranging from physical concerns to abstract concerns.

In robotics control, automation is desirable, because of the repetitious, fatiguing, and hazardous mechanical operations involved. Then the issue of ‘agile manufacturing’ can emerge, in which manufacturers are able to respond quickly to the need for high-quality customized products. Sometimes, remote operators have to supervise behaviour of a group, not directly but by ‘encouraging or exorting’ the desired behavior of the group. This is called ‘hortatory control’, in which the systems that are being controlled require a high degree of autonomy. An example of this is road traffic controllers, which try to influence the flow of traffic in an area around a city by informing travellers of current and expected road conditions and encouraging them to take certain actions. The biggest challenge in this is to provide information that is the most effective in attracting users to adopt certain behavior.

What can be concluded?

Automation has been beneficial to safety, comfort, and job satisfaction, but it also has lead to problems. Therefore, automation should be carefully designed with consideration of the role of the human.