What are the principles of response selection?

The difficulty and speed of selecting a response or an action is influenced by different factors. Five of them are important for system design: decision complexity, expectancy, compatibility, the speed-accuracy tradeoff, and feedback.

Decision complexity

How fast an action is selected is influenced by how many alternative actions there are possible. This is referred to as the complexity of the decision. For example, each action of the Morse code operator has only two alternatives (dit or dah) and is simpler than the choice of a typist, who has to choose between 26 letters. Thus, the Morse code operator can generate a greater number of keystrokes per minute, and users can select an action faster because there are only two options.

This relationship between response selection time and decision complexity is explained by the Hick-Hyman law of reaction time (RT). When reaction time or response time is plotted as a function of Log2(N) instead of (N), the function is linear. According to the Hick-Hyman law, humans process information at a constant rate.

This law does not suggest that systems designed to make simpler decisions are better. Instead, if an user has to transmit information, it seems to be more efficient to do this by a smaller number of complex decisions than a large number of simple decisions. This is called the ‘decision complexity advantage’. As an example, think of a typist who can convey the same message faster than the Morse code operator. Even though the keystrokes are made more slowly, there are fewer keystrokes. This means that ‘shallow menus’ with many items are better than ‘deep menus’ with just a few items.

Response expectancy

When we expect information, we perceive it more rapidly. We also select actions more rapidly and accurately when we expect to carry these actions out. For example, we do not expect a car in front of us to stop suddenly. This means that in this case we are slower to apply the brake than we would be when the light turns yellow or red at an intersection, in which we expect it.

Compatibility

Stimulus-response compatibility describes the relationship between the location of a control or movement of a control response and the location or movement of the stimulus or display to which the control is related. There are two subprinciples that characterize a compatible mapping:

1. Location compatibility. This means that the control location should be close to the entity being controlled or the display of that entity.
2. Movement compatibility. This means that the direction of movement of a control should be congruent with the direction both of movement of the feedback indicator and of the system movement itself.

The speed-accuracy tradeoff

Factors that make the selection of a response longer (complex decisions, unexpected actions, incompatible responses) also lead to more errors. There is thus a positive relationship between response time and error rates, or a positive relationship between speed and accuracy. However, in this relationship, there is no tradeoff. However, if we try to execute actions in a fast way, we are more likely to make errors. If we have to be very cautious, we will eb slow. Then, there is a negative correlation or a speed-accuracy tradeoff.

Feedback

Most of the controls and actions that we take are associated with visual feedback which indicates the system response to the control input. In a car, there is the speedometer which offers visual feedback from the control of the accelerator. A good control design should also include more direct feedback, for example the resistance on a stick as it is moved. The feedback can be auditory (the beep of a phone) or visual (a light next to a switch show). Feedback is said to be good, when it is direct or instantaneous. When feedback is delayed, even by 100 msec, it can be harmful and especially when the operator is less skilled or if the feedback cannot be filtered out by selective attention mechanisms. One example of a harmful delayed feedback is while talking on a radio or telephone.

What is discrete control activation?

One way to make controls less susceptible to errors and delays is to make the controls early visible. In addition, there are other design features that also make the activation of controls less susceptible to errors and delays.

Physical feel

As said, feedback is a positive feature of controls. In some controls, there is more feedback than in others. For example, the toggle switch has good feedback: the state changes in a clear visual manner and there is also an auditory click and a tactile snap (a loss of resistance) when it moves into a new position. For other types of discrete controls, one should focus on how to provide feedback. Some have touch screens, which do not work well. Push-button phones that lack an auditory beep following the keypress also do not do this well. Feedback lights should be complemented with other indications of state change, and visual feedback should be immediate.

Size. Smaller keys are problematic: they can lead to errors when people accidentally press multiple keys at the same time.

Confusion and Labeling. Keypresses can also lead to errors when the identification of a key is not well specified or when it is new to the user (when someone does not know which location to touch). These errors are more likely to happen when large sets of identically appearing controls are unlabeled or poorly labelled, and when labels are physically displaced from their associated controls.

What about positioning control devices?

Positioning or pointing something refers to for example moving a cursor to a point on a screen, reaching with a robot arm to grab an object, or move the frequency of a radio to a new frequency. There are control devices such as the mouse, joystick for these goals. The authors now describe the relationship between the movement of an entity (the cursor) to a destination (a target). They describe a model that accounts for the time to make such movements.

Movement time

Control requires two types of movements:

1. Movement required for the hands or fingers to reach the control (grab the mouse)
2. Moving the control in some direction (positioning the cursor)

These movements take time. These times can be predicted by a model called the Fitt’s law: MT = a + b log2(2A/W). In this formula, a = amplitude of the movement, W = width of the target or the desired precision with which the cursor must land. Movement time is then linearly related to the logarithm of the term (2A/W), which is an index of difficulty of the movement.

Device characteristics

There are four categories of control devices that are used for positioning or pointing: direct position controls (light pen and touch screen), in which the position of the human hand/finger corresponds with the desired location of the cursor. Second, there are indirect position controls (the mouse), in which the hands are on a different location than on the screen, but with the hands they move the mouse to point somewhere. Third, there are indirect velocity controls such as the joystick and the cursor keys. This means that control in a given direction leads to velocity of cursor movement in that direction. For example, for cursor keys, this means repeated presses or holding it down for a longer period of time. Joysticks can be of three types: isotonic (moved freely and will rest wherever they are positioned), isometric (rigid but produce movement proportional to the force that is applied), and spring-loaded (offer resistance proportional to the force applied and the amount of displacement, springing back to the neutral position when pressure is released). The fourth category of control devices is voice control.

There are two important variables which affect usability of controls for pointing:

1. Feedback of the current state of the cursor should be salient, visible, and immediate.
2. Performance is activated in a more complex way by the system ‘gain’.

Gain is described by: G = (change of cursor)/(change of control position.

A high-gain device is then one in which a small displacement of the control leads to a large movement of the cursor or produces a fast movement in the case of a velocity control device. The gain of direct position controls (touch screen and light pens) will be 1.0. It seems that the ideal gain for indirect control devices should be in the range of 1.0 to 3.0.

What is task performance dependence?

It seems that the two best control devices are the two direct position controls and the mouse. When there are more complex spatial activities such as drawing or handwriting, indirect positioning devices seem to be the best in providing natural feedback. Cursor keys are adequate for some tasks, but they do not produce long movements such as during text editing.

What is the work space environment?

Devices are often used within a broader workspace, such as a display. Display size is also important. Greater display size need high-gain devices. Smaller displays require precise manipulation and lower gain. Vertically mounted displays are also of influence: these impose greater costs on direct positioning devices.

What are verbal and symbolic input devices?

For symbolic, numerical, or verbal information that is involved in system interaction, keyboards or voice control are often the interfaces of choice.

Numerical data entry

For numerical data, numerical keypads or voice control are the best. Voice control is the most compatible and natural, but it has technologies problems which slow the rate of possible input. Keypads come in three forms: the linear array (above the keyboard). This is not preferred, because it costs a lot of time to move from key to key. Then there is the 3*3 square arrays, this reduces movement time. Some research suggests that the layout with 123 on the top (such as on the telephone) is better than the 789 on top (calculator, laptop keyboard), but not that better that the 789 keyboards should be redesigned.

Linguistic data entry

The computer keyboard is the commonly used device for linguistic data. An alternative to the standard QWERTY layout is the chording keyboard. In a chording keyboard, individual items of information are entered by simultaneous depression of combinations of keys. This has three advantages:

1. The hands never need to leave the chord keyboard, and there is no requirement for visual feedback to monitor the correct placement of a thumb or finger.
2. The chording board is less susceptible to repetitive stress injury or carpal tunnel syndrome.
3. After extensive practice, chording keyboards support more rapid word transcription processing than the standard type-writer keyboard, because there is no movement-time trade-off.

However, before one can use a chording keyboard, extensive learning is required.

What about voice input?

The benefits of voice control

Voice is a natural form of communication. This naturalness could be employed in many control interfaces. The benefits are especially clear in dual-task situations. For example, when the hands are busy with other tasks (driving the car), it would be handy if someone could talk to the interface. For example, dialling by voice command is handy.

Costs of voice control

There are four distinct costs of voice control:

Confusion and Limited Vocabulary Size. Voice recognition systems are prone to make errors, because they could classify similar-sounding utterances as the same (cleared to vs. cleared through).

Constraints on Speed. The natural flow of speech does not necessarily place pauses between different words. Then, the computer does not know when to ‘stop counting syllables’ and determine the end of a word. This may require the speaker to speak in a slow way, with pausing between each word. There is also a lot of time required to ‘train’ voice systems to understand the individual speakers’ voice.

Acoustic Quality and Noise and Stress. A noisy environment hinders the voice control system. Also, under conditions of stress, one’s voice can change. Therefore, there should be great caution when designing voice control systems that are used as a part of emergency procedures.

Compatibility. Voice control is less suited for controlling continuous movement than most of the available manual devices. For example, it is easier to steer a car by manually controlling the steering wheel compared to saying ‘a little left, now a little more left’.

What about continuous control and tracking?

Sometimes we need to make a cursor or a vehicle follow a ‘track’ or ‘continuously moving dynamic target’.

The Tracking Loop: Basic Elements

In a tracking task, there are basic elements. Each element receives a time-varying input and produces a corresponding time-varying output. Every signal in the tracking loop is represented as a function of time, f(t). When driving a car, the human operator perceives a discrepancy or error between the desired state of the vehicle and its actual state. Then the driver wants to reduce this error function of time, e(t). To do so, he or she uses force, f(t) to control the wheels. This produces a rotation, u(t) of the steering wheel, and this produces control output. The relationship between the force and the steering wheel is called the ‘control dynamics’. The movement of the wheel to a given time function u(t) is called the system output, o(t). When presented on a screen, this output position is called the cursor. The relationship between control output u(t) and system response o(t) is defined as the system dynamics.

What about the input?

Examples of tracking tasks are drawing a straight line on a piece of paper, or driving a car down a straight road on a windless day. In both these cases, there is a command target input and a system output. However, the input does not vary: after you get the original course set, there is nothing to do but to move forward. You can drive fast or drive slow. But, when the road is curvy, one needs to make corrections and there is uncertainty. Then, error and workload can increase if you try to move faster. The frequency with which corrections must be made are called ‘the bandwidth of input’. In tracking tasks, this bandwidth is expressed in terms of the cycles per second (Hz) of the highest input frequency present in the command or disturbance input. When bandwidth is above 1 Hz, it is hard for people to perform tracking tasks. In most systems, the bandwidth is about 0.5 Hz. Higher bandwidth means higher complexity. This complexity is based on the order of a control system.

Control order

Position control. The order of a control system refers to whether a change in the position of the control device leads to a change in the position (zero-order), velocity (first-order), or acceleration (second-order) of the system output. For example, moving a pen across a paper leads to a new position of the system output. If you hold your pen still, the system output is also still. This is called zero-order control.

Velocity control. Think of a digital car radio. When you depress the button to position the radio, this creates a constant rate of change (velocity) of the frequency setting. For some devices, pressing the button harder leads to a proportionally greater velocity. This is called first-order control. Most pointing devices use velocity control: the greater a joystick is deflected, the faster will be the cursor motion. Another example of first-order control is the position of the steering wheel (input) and the rate of change (velocity) of heading your car (output).

Acceleration control. In a spacecraft there is inertia, and each rocket thrust produces an acceleration of the craft for as long as the engine is firing. This is called a second-order acceleration control system. Another example of this rolling a pop can to a new position or command input on a board. Second-order systems are often difficult to control, because they are sluggish and instable. Therefore, they are rarely designed into systems. Second order systems can only be successfully controlled if the tracker anticipates, inputting a control now for an error that will be predicted to occur in the future.

What about time delays and transport lags?

Higher-order systems (second-order) have lags. For example, when navigating through virtual environments, there is often a delay between movement of the control device and the position. These delays are called transport lags and they also require anticipation, which leads to higher human workload and can lead to system errors.

What is stability?

Next to lag, gain, and bandwidth, stability is also an important factor for control systems. When there is instability of control, this is called closed-loop instability, or negative feedback instability. Closed-loop instability results from three factors:

1. There is a lag in the total control loop from the system lag or from the human operator’s response time.
2. The gain is too high.
3. The human is trying to correct an error too fast and is not waiting until the lagged system stabilizes before applying another input.

Human factor engineers can offer five solutions which can be implemented to reduce closed-loop instability:

1. Lower the gain
2. Reduce the lags
3. Caution the operator to change strategy so that he or she does not try to correct every input but filters out the high-frequency ones, thus reducing the bandwidth
4. Change strategy to seek input that can anticipate and predict
5. Change strategy to go ‘open loop’

What is open loop?

When the operator perceives an error and tries to correct it, the loop is called ‘closed’. However, it can also be the case that the operator knows where the system output needed to be and responded with precise correction to the control device to produce the goal. Then, the loop is ‘open’: the operator does not need to perceive the error and will not be looking at the system output. However, open-loop behavior depends on the operator’s knowledge of:

1. Where the target will be and;
2. How the system output will respond to his or her control input.

What is remote manipulation / telerobotics?

Sometimes, direct human control is desirable, but not feasible. One example is remote manipulation, for example when operators control an underseas explorer of an unmanned air vehicle (UAV). The second one is hazardous manipulation, such as when working with highly radioactive material. This is called ‘telerobotics’. Telerobotics comes with challenges because of the absence of direct viewing. The goal of the designer of such systems is to create a sense of ‘telepresence’: a sense that the operator is actually immersed within the environment and is directly controlling the manipulation as an extension of his or her arms and hands. There are different factors that prevent this goal from being achieved, which are discussed last.

Time delay

Systems often involve time delays between manipulation and visual feedback for the controller. These delays can present challenges for effective control.

Depth perception and Image Quality

Teleoperation requires tracking or manipulating in three dimensions. However, human depth perception in 3-D displays is often less adequate for precise judgment. One solution for this may be the implementation of stereo. However, a problem with stereo implementation might be that two cameras must be mounted and two separate dynamic images must be transmitted over what may be a limited bandwidth channel.

Proprioceptive Feedback

In addition to visual feedback, proprioceptive or tactile feedback is also important. Consider for example what happens when a remote manipulator punctures a container of radioactive material by squeezing too hard. To prevent such accidents, designers would like to present the same tactile and proprioceptive sensations of touch, feel, pressure, and resistance that we experience as our hands grasp and manipulate objects directly. To prevent such accidents, designers need to present the same tactile and proprioceptive sensations of touch, feel, pressure, and resistance that we experience as our hands grasp and manipulate objects directly. But it is challenging to present such feedback effectively.

What are the solutions?

The biggest problem in teleoperator systems is the time delay. An effective solution would then be to reduce the delay. Sometimes this involves reducing complexity. A second solution might be to develop predictive displays that are able to anticipate future motion and position of the manipulator on the basis of present state and the operator’s current control actions and future intentions. These tools are useful, but they are only as effective as the quality of the control laws of system dynamics that they embody. Furthermore, the system cannot achieve effective predictions of a randomly moving target. A third solution to avoid delayed feedback is by implementing a computer model of the system dynamics, allowing the operator to implement the required manipulation in ‘fast time’.