Control of behaviour by competing learning systems - summary of chapter 11 of The Wiley Handbook of cognitive control

The Wiley Handbook of cognitive control
Chapter 11
Control of behaviour by competing learning systems

Introduction

Most of us struggle sometimes to control our behaviour. Habit learning may play a role in this ‘intention-behaviour’ gap.

A dual-system theory of action control holds that these learning processes sometimes compete, but also cooperate, to control action.

Defining goal-directed action and habits

The term goal-directed actions refers to instrumental behaviours that are only performed when one has a certain goal and believes that this behaviour will increase the likelihood of reaching that goal.

Instrumental behaviours are learned as a consequence of a causal relationship between the action or response (R) and its outcome (O), as opposed to being controlled purely by predictive Pavlovian relationships between environmental stimuli (S) and the outcome. Once the instrumental status of behaviour has been established, we need to assess whether it meets the belief and desire criteria of goal-directed action. The belief criterion stipulates that goal-directed actions are mediated by knowledge of the causal action-outcome relationship. The desire criterion holds that goal-directed actions are only executed when the outcome is currently desirable, they need to constitute a goal.

When a goal-directed action is repeatedly performed, it can gradually turn into a habit, which is triggered by contextual stimuli, even when the consequences are no longer valuable.

Thorndike proposed that the experience of reward following an instrumental response leads to the strengthening of a mental association between contextual stimuli (S) and the response (R). On future occasions, the context will directly activate the response through the S-R association. The omission of a dreaded aversive event may also act to strengthen the S-R link. The occurrence of an aversive outcome should weaken the S-R association.

The difference between habits and goal-directed actions is that only in the latter case the performance is mediated by knowledge of the R à O relationship and an evaluation of the anticipated outcome in light of one’s current motivation. Habits are mediated by S-R links. They are ‘behaviourally autonomous’ of the current desirability of the outcome.

Habits are considered to have adaptive value. They can be executed fast in an efficient manner. By freeing up other cognitive resources, habit formation allows us to attend to other important matters.

Investigating the goal-directed versus habitual status of behaviour

Frequent repetition leads to a shift from goal-directed control towards habit. Habits are contextually dependent.

Outcome-revaluation studies in animals

Goal-directed action in animals

Rats are capable of goal-directed action.

Habit in animals

Behavioural repetition renders behavioural habitual in the sense that it is no longer dependent on the current desirability of the outcome.

Outcome-revaluation studies in humans

Goal-directed action in humans

It is generally assumed that humans are capable of goal-directed action.

Habit in humans

Human behaviour loses its immediate sensitivity to outcome value after extensive repetition.

Neurobiology of action and habit

Animal lesioning research

Animals studies have implicated several regions in goal-directed control.

Pretraining lesions of the prelimbic cortex lead to behavioural autonomy, even after limited training. Therefore, the prelimbic cortex appears to be crucially involved in the R-O learning processes that underlies goal-directed action.

Both pre- and post-training lesions of the dorsomedial striatum disrupts goal-directed performance. This implicates this subcortical area in both the acquisition and expression of goal-directed action. It has been suggested that this region is responsible for long-term storage of R-O associations. These cortical and striatal areas are heavily anatomically interconnected, suggesting that they are part of a corticostriatal pathway for goal-directed action. Another component of this pathway is the mediodorsal thalamus.

Lesions of the dorsolateral striatum disrupts the formation of habits. R-O associations remain to some extent intact following extensive training, but may be by-passed by the more direct S-R habits. The ability to act in line with current goals can be compromised by concurrently formed, competing habits.

An important area for habitual control is the infralimbic cortex. The infralimbic cortex and the dorsolateral striatum may be part of a corticostriatal pathway for habitual action control.

Human neuroimaging research

Human neuroimaging research has provided evidence for dissociable corticostriatal pathways that underlie goal-directed versus habitual action control. Preferential responding for the still-valuable outcome is associated with activity in the ventromedial prefrontal cortex/medial orbitofrontal cortex. This maybe the functional homologue of the rodent prelimbic cortex.

Clinical relevance

The gradual formation of S-R habits may contribute to treatment resistance of frequently performed maladaptive behaviours. A tendency towards aberrantly strong habit formation and/or weak goal-directed control may render certain individuals particularly vulnerable to the development of maladaptive habits.

Addiction

Drug abuse was the earliest clinical condition to be characterized in terms of dominant S-R habitual control. Recreational drug use starts out as voluntary, but the highly rewarding effect of drugs can lead to the formation of strong S-R associations, such that goal-directed drug seeking rapidly transits into stimulus-driven compulsive drug-seeking habits.

The gradual shift towards behavioural autonomy of the drug-seeking response is reflected in the progression from ventral to dorsal striatum. Weak prefrontal functioning as a vulnerability factor as well as a consequence of prolonged drug use has been proposed to underlie impaired goal-directed and inhibitory control.

Obsessive-compulsive disorder

There is a general tendency toward habit propensity in OCD. This is related to hyperactivity in the ventromedial prefrontal-orbitorfrontal cortex and caudate.

Obesity and eating disorders

Force of habit may help explain why some people fail to break unhealthy habits. These individuals may show accelerated habit formation.

Habits as a transdiagnostic compulsive trait

The habit propensity may be a transdiagnostic trait for compulsive conditions. The question arises to what extent habit propensity is characteristic of compulsive conditions specifically, or instead is a common factor across psychopathologies and in individuals with impaired prefrontal functioning.

Stress may lead to dominant habitual control.