The Minor Puzzle about Habitual Processes

A rat has been given food contingent on its pressing a level. When it presses the lever, is its action habitual or instrumental? By the end of this section you should understand why this question is puzzling and also how to resolve the puzzle. You should also understand devaluation, and be able to understand an experiment that provides some of the foundational evidence for the dual-process theory of instrumental action.

youtube
stream
slides only

This recording is also available on stream (no ads; search enabled). Or you can view just the slides (no audio or video). You should not watch the recording this year, it’s all happening live (advice).

If the video isn’t working you could also watch it on youtube. Or you can view just the slides (no audio or video). You should not watch the recording this year, it’s all happening live (advice).

If the slides are not working, or you prefer them full screen, please try this link.

The recording is available on stream and youtube.

Notes

You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often. This indicates that the rat’s lever pressing is an instrumental action, for manipulating the outcome of the action has changed its frequency. But is this lever pressing primarily a consequence of habitual processes?

In thinking about this question, consider how we characterised habitual and goal-directed processes (in Goal-Directed and Habitual Processes). What does the hypothesis that the rat’s lever pressing is dominated by habitual processes predict? And what does the alternative hypothesis that the rat’s lever pressing is dominated by goal-directed processes predict?

Because the aim of this section is to get you thinking about the questions, these notes do not answer them. The recording will take you through some considerations.

The Minor Puzzle

Dickinson (1985) found that when a rat has learned to perform an instrumental action to obtain a food and when the food is devalued, the frequency with which the rat performs the action is reduced but the rat does nevertheless continue to perform the action

If the action is habitual, why is it influenced by devaulation at all?

If the action not habitual but controlled by goal-directed processes, why does it still occur (albeit less frequently) after devaluation?

The Dual-Process Theory of Instrumental Action Revisited

As the term ‘habitual’ is used on this course, actions are the wrong kind of thing to be described as habitual. It is the processes that trigger and guide actions that can be habitual, not the actions themselves.

This matters because on the dual-process theory of instrumental action, one action may be simultaneously guided by two or more distinct kinds of process, one goal-directed and the other habitual.

The Minor Puzzle is telling us that, in the case of the rat’s action, both kinds of processes are influential. The frequency with which the rat performs the action is reduced, indicating that it is influenced by goal-directed processes but the rat does nevertheless continue to perform the action, indicating that it is influenced by habitual processes.

Conclusion

Actions are controlled by two or more distinct kinds of process, one goal-directed and the other habitual. If an action were very strongly dominated by habitual processes, we might informally label the action ‘habitual’. But, as we will further explore in Goal-Directed and Habitual: Some Evidence, the actions of humans, like rats, are often significantly influenced by both kinds of process.

Appendix: Reflexes

Reflexes are an example of instrumental actions whose occurrence is fully explained neither by the habitual nor by the goal-directed process:

‘A light puff of air directed at the cornea makes the eye blink. A tap just below the knee causes the leg to kick. A loud noise causes a startle reaction. These are all examples of reflexes. A reflex involves two closely related events: an eliciting stimulus and a corresponding response. Furthermore, the stimulus and response are linked. Presentation of the stimulus is followed by the response, and the response rarely occurs in the absence of the stimulus.’ (Domjan, 2010, p. 30)

Ask a Question

Your question will normally be answered in the question session of the next lecture.

More information about asking questions.

Glossary

devaluation : To devalue some food (or video clip, or any other thing) is to reduce its value, for example by allowing the agent to satiete themselves on it or by causing them to associate it with an uncomfortable event such as an electric shock or mild illness.

dual-process theory of instrumental action : Instrumental action ‘is controlled by two dissociable processes: a goal-directed and an habitual process’ (Dickinson, 2016, p. 177). (See instrumental action.)

extinction : In some experiments, there is a phase (usually following instrumental training and an intervention such as devaluation) during which the subject encounters the training scenario exactly as it was (same stimuli, same action possibilities) but the actions produce no revant outcomes. In this extinction phase, there is no reward (nor punishment). (It is called ‘extinction’ because in many cases not rewarding (or punishing) the actions will eventually extinguish the stimulus--action links.)

goal-directed process : A process which involves ‘a representation of the causal relationship between the action and outcome and a representation of the current incentive value, or utility, of the outcome’ and which influences an action ‘in a way that rationalizes the action as instrumental for attaining the goal’ (Dickinson, 2016, p. 177).

habitual process : A process underpinning some instrumental actions which obeys Thorndyke’s Law of Effect: ‘The presentation of an effective [=rewarding] outcome following an action [...] reinforces a connection between the stimuli present when the action is performed and the action itself so that subsequent presentations of these stimuli elicit the [...] action as a response’ (Dickinson, 1994, p. 48). (Interesting complication which you can safely ignore: there is probably much more to say about under what conditions the stimulus–action connection is strengthened; e.g. Thrailkill, Trask, Vidal, Alcalá, & Bouton, 2018.)

instrumental action : An action is instrumental if it happens in order to bring about an outcome, as when you press a lever in order to obtain food. (In this case, obtaining food is the outcome, lever pressing is the action, and the action is instrumental because it occurs in order to bring it about that you obtain food.)
You may encounter variations on this definition of instrumental in the literature. For instance, Dickinson (2016, p. 177) characterises instrumental actions differently: in place of the teleological ‘in order to bring about an outcome’, he stipulates that an instrumental action is one that is ‘controlled by the contingency between’ the action and an outcome. And de Wit & Dickinson (2009, p. 464) stipulate that ‘instrumental actions are learned’.

References

Anscombe, G. E. M. (1957). Intention. Oxford: Blackwell.

Bach, K. (1978). A representational theory of action. Philosophical Studies, 34(4), 361–379. https://doi.org/10.1007/BF00364703

Davidson, D. (1980). Essays on actions and events. Oxford: Oxford University Press. https://doi.org/10.1093/0199246270.001.0001

de Wit, S., & Dickinson, A. (2009). Associative theories of goal-directed behaviour: A case for animalhuman translational models. Psychological Research PRPF, 73(4), 463–476. https://doi.org/10.1007/s00426-009-0230-6

Dickinson, A. (1985). Actions and habits: The development of behavioural autonomy. In L. Weiskrantz (Ed.), Animal intelligence. Oxford: Clarendon Press.

Dickinson, A. (1994). Instrumental conditioning. In N. Mackintosh (Ed.), Animal learning and cognition. London: Academic Press.

Dickinson, A. (2016). Instrumental conditioning revisited: Updating dual-process theory. In J. B. Trobalon & V. D. Chamizo (Eds.), Associative learning and cognition (Vol. 51, pp. 177–195). Edicions Universitat Barcelona.

Domjan, M. (2010). The Principles of Learning and Behavior. Wadsworth Cenage Learning.

Kalis, A., & Ometto, D. (2021). An Anscombean Perspective on Habitual Action. Topoi, 40(3), 637–648. https://doi.org/10.1007/s11245-019-09651-8

Keramati, M., Smittenaar, P., Dolan, R. J., & Dayan, P. (2016). Adaptive integration of habits into depth-limited planning defines a habitual-goaldirected spectrum. Proceedings of the National Academy of Sciences, 113(45), 12868–12873. https://doi.org/10.1073/pnas.1609094113

Shepherd, J. (2021). The Shape of Agency: Control, Action, Skill, Knowledge. The Shape of Agency. Oxford: Oxford University Press.

Thrailkill, E. A., Trask, S., Vidal, P., Alcalá, J. A., & Bouton, M. E. (2018). Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior. Journal of Experimental Psychology: Animal Learning and Cognition, 44, 370–384. https://doi.org/10.1037/xan0000188

Velleman, D. (2000). The possibility of practical reason. Oxford: Oxford University Press.