Link Search Menu Expand Document

Goal-Directed and Habitual Processes

This section introduces a key distinction between goal-directed and habitual processes.

This recording is also available on stream (no ads; search enabled). Or you can view just the slides (no audio or video). You should not watch the recording this year, it’s all happening live (advice).

If the video isn’t working you could also watch it on youtube. Or you can view just the slides (no audio or video). You should not watch the recording this year, it’s all happening live (advice).

If the slides are not working, or you prefer them full screen, please try this link.

The recording is available on stream and youtube.

Notes

In Instrumental Action, we asked about the relation between an instrumental action and the outcome or outcomes to which it is directed. As we saw, the Standard Answer given by philosophers is that intention grounds this relation.

But are there maybe things other than intentions which might link an instrumental action to an outcome?

A Clue from Animal Learning

According to Dickinson (2016, p. 177):

‘instrumental behavior is controlled by two dissociable processes: a goal-directed and an habitual process’

He goes on to specify what the ‘goal-directed process’ involves:

‘an action is goal-directed if it is mediated by the interaction of a representation of the causal relationship between the action and outcome and a representation of the current incentive value, or utility, of the outcome in a way that rationalizes the action as instrumental for attaining the goal’ Dickinson (2016, p. 177).

Dickinson’s ‘goal-directed process’ corresponds to the belief--desire model we just considered. The ‘representation of the causal relationship between the action and outcome’ could be a belief about which action will bring an outcome about (e.g. the belief that if I pour, I will fill Zak’s glass). And the ‘representation of the current incentive value, or utility, of the outcome’ could be a desire.

philosophy animal learning decision theory
belief representation of the causal relationship between the action and outcome subjective probability
desire representation of the current incentive value, or utility, of the outcome preference

Table: rough correspondence between terms used for modelling action across three disciplines.

So when Dickinson says that instrumental actions are ‘controlled by two dissociable processes’, he is implying that the Standard Answer about belief, desire and intention cannot fully explain the relation between an instrumental action and the outcome or outcomes to which it is directed. If he is right, we also have to consider something he calls ‘an habitual process’.

What Are Habitual Processes?

Habitual processes involve connections between stimuli and actions. For example, the presence of an empty glass (a stimulus) may be connected to the action of pouring. These connections are characterised by two features:

  1. When the action is performed in the presence of the simulus, the connection between action and stimulus is strengthened (or ‘reinforced’) if the action is rewarded.

  2. If the connection is strong enough, the presence of the stimulus will cause the action to occur.

This is another way of stating Thorndyke’s Law of Effect:

‘The presenta­tion of an effective [=rewarding] outcome following an action [...] rein­forces a connection between the stimuli present when the action is per­formed and the action itself so that subsequent presentations of these stimuli elicit the [...] action as a response’ (Dickinson, 1994, p. 48).

How do habitual processes differ from those involving belief, desire and intention? Two differences are important for our purposes:

  1. The effects of habitual processes do not depend on what you currently desire. This is because the strength of the stimulus–action connection depends only on what was rewarding for you in the past, not what is rewarding for you now.

  2. The effects of habitual processes do not depend on what you currently believe about which outcome the action will have. This is because the strength of the stimulus–action connection depends only on what outcomes the action had in the past, not on which outcomes it will have now.

Because habitual processes have these features, we can be sure that they are genuinely distinct from processes involving belief, desire and intention.

Habitual Processes and Instrumental Action

Our Main Question is, What is the relation between an instrumental action and the outcome or outcomes to which it is directed? This question can be answered by invoking habitual processes. For if an action is due to an habitual process, then there is a stimulus–action connection which caused it. This stimulus–action connection must have been strengthened in the past because, often enough, some (one or more) rewarding outcomes occurred when the action was performed in the presence of the stimulus. But since habitual processes exist to enable the agent repeatedly bring about such rewarding outcomes, it follows that the action occurs now in order to bring about these (one or more) rewarding outcomes. That is, the action is directed to the outcome; it is an instrumental action.

The Standard Answer therefore fails to provide a full answer to the Main Question about instrumental action. To fully answer it we need not only belief, desire and intention but, minimally, also the kind of stimulus–action connections involved in habitual processes.

So What?

After this section, you should understand what an instrumental action is, you should understand the Main Question, and you should understand how habitual processes and goal-directed processes differ.

The next step is to investigate possible consequences for philosophical theories of action.

Ask a Question

Your question will normally be answered in the question session of the next lecture.

More information about asking questions.

Glossary

directed : For an action to be directed to an outcome is for the action to happen in order to bring that outcome about.
goal-directed process : A process which involves ‘a representation of the causal relationship between the action and outcome and a representation of the current incentive value, or utility, of the outcome’ and which influences an action ‘in a way that rationalizes the action as instrumental for attaining the goal’ (Dickinson, 2016, p. 177).
habitual process : A process underpinning some instrumental actions which obeys Thorndyke’s Law of Effect: ‘The presenta­tion of an effective [=rewarding] outcome following an action [...] rein­forces a connection between the stimuli present when the action is per­formed and the action itself so that subsequent presentations of these stimuli elicit the [...] action as a response’ (Dickinson, 1994, p. 48). (Interesting complication which you can safely ignore: there is probably much more to say about under what conditions the stimulus–action connection is strengthened; e.g. Thrailkill, Trask, Vidal, Alcalá, & Bouton, 2018.)
instrumental action : An action is instrumental if it happens in order to bring about an outcome, as when you press a lever in order to obtain food. (In this case, obtaining food is the outcome, lever pressing is the action, and the action is instrumental because it occurs in order to bring it about that you obtain food.)
You may encounter variations on this definition of instrumental in the literature. For instance, Dickinson (2016, p. 177) characterises instrumental actions differently: in place of the teleological ‘in order to bring about an outcome’, he stipulates that an instrumental action is one that is ‘controlled by the contingency between’ the action and an outcome. And de Wit & Dickinson (2009, p. 464) stipulate that ‘instrumental actions are learned’.
model : A model is a way some part or aspect of the world could be.
model based : A model-based process is one that relies on a model. This is usually thought to involve deriving predictions from representations of the model. Compare Dayan & Berridge (2014, p. 477): ‘A model-based strategy involves prospective cognition, formulating and pursuing explicit possible future scenarios based on internal representations of stimuli, situations, and environmental circumstances.’
model free : A model-free process is one that does not rely on a model. This term is often used for processes which exploit causal or statistical connections that are not represented.
outcome : An outcome of an action is a possible or actual state of affairs.
stimulus : A stiumlus is just a situation or event. Typically, ‘stimlus’ is used to label things which do, or might, prompt actions such as the presence of a lever or the flashing of a light.

References

Dayan, P., & Berridge, K. C. (2014). Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation. Cognitive, Affective, & Behavioral Neuroscience, 14(2), 473–492. https://doi.org/10.3758/s13415-014-0277-8
de Wit, S., & Dickinson, A. (2009). Associative theories of goal-directed behaviour: A case for animalhuman translational models. Psychological Research PRPF, 73(4), 463–476. https://doi.org/10.1007/s00426-009-0230-6
Dickinson, A. (1994). Instrumental conditioning. In N. Mackintosh (Ed.), Animal learning and cognition. London: Academic Press.
Dickinson, A. (2016). Instrumental conditioning revisited: Updating dual-process theory. In J. B. Trobalon & V. D. Chamizo (Eds.), Associative learning and cognition (Vol. 51, pp. 177–195). Edicions Universitat Barcelona.
Thrailkill, E. A., Trask, S., Vidal, P., Alcalá, J. A., & Bouton, M. E. (2018). Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior. Journal of Experimental Psychology: Animal Learning and Cognition, 44, 370–384. https://doi.org/10.1037/xan0000188

Endnotes