Link Search Menu Expand Document

Preference vs Aversion: A Dissociation

Your preferences can be incompatible with your aversions (and thereby with primary motivational states). This shows that there is not a single system of preferences in rats or humans.

This recording is also available on stream (no ads; search enabled). Or you can view just the slides (no audio or video). You should not watch the recording this year, it’s all happening live (advice).

If the video isn’t working you could also watch it on youtube. Or you can view just the slides (no audio or video). You should not watch the recording this year, it’s all happening live (advice).

If the slides are not working, or you prefer them full screen, please try this link.

The recording is available on stream and youtube.

Notes

There are at least two kinds of motivational state, which have distinct roles in explaining behaviour.

One is desires or preferences. These are states that can be influenced by fashion and all kinds of learning.

The other is primary motivational states. These include hunger, thirst, satiety, aversion and disgust. They are linked to biological needs and not always learned. Although they can be modified by learning, there are limits on the influence of learning.

How are primary motivational states and desires related?

Consider two views:

Wrong View Primary motivational states and preferences form a single system for guiding action selection. Unless something goes badly wrong, your primary motivational states inform your preferences directly. To illustrate, if you are hungry for a food, you desire it; and if you are averse to a food, you do not desire it.

In this section we consider evidence against the Wrong View and in favour of:

Correct View There are at least two kinds of motivational state which have distinct roles in explaining behaviour. If primary motivational states inform your preferences at all, they do so only indirectly. You may hunger for a food you do not desire to eat, and you may desire to eat a food to which you are averse.[1]

Anecdote: Dickinson’s Water Melons

Dickinson ate melons for the first time. Shortly after consumed red wine and suffered mild toxicosis. Waking up the next day, Dickinson felt thirsty.

‘once again in need of refreshment, he readily retraced the route to the water melon stall. But when confronted with the direct signal for approach, the sight of those sliced, juicy, rosy-red segments which had looked and were so refreshing on the previous occasion, his appetite abated. And indeed, when he managed to take a bite, he discovered that he had an aversion to water melon with the consequence that none has passed his lips since that day.’ (Dickinson & Balleine, 1993, p. 285)

The aversion makes sense because becoming unwell after eating a novel food can cause aversion to it (Domjan, 2010, p. 71).

Puzzle: if Dickinson was not already averse to melon when he work up, how could eating melon cause aversion? If Dickinson was already averse, why did he set out to eat melon?

Does the Correct View (see above) suggest a solution to this puzzle?

The Questions

Can your primary motivational states diverge from your preferences?

In particular:

  1. Can hunger drive you to seek a novel food even tho you have no desire to eat it? And can satiety reign in your search for a food even though you desire to eat it?
    Yes (Balleine, 1992)!

  2. And can sugar solution rank highly among your preferences even after you have become averse to it?
    Yes (Balleine & Dickinson, 1991)!

The evidence for positive answers to these questions supports the Correct View (see above): Preferences and primary motivational states have distinct roles in explaining behaviour. If primary motivational states inform your preferences at all, they do so only indirectly.

Conclusion: your behaviours are driven by two (or more) motivational systems which are, to an interesting degree, independent of each other.

This conclusion gives rise to an interface problem ...

An Interface Problem

We have seen evidence for these claims:

  1. Primary motivational states guide some actions.

  2. Preferences guide some actions.

  3. Pursuing a single goal can involve both kinds of state, as, for instance, when the rat obtains food by pressing the lever and entering the magazine to retrieve it.

  4. Primary motivational states can differ from preferences, as, for instance, when the rat is hungry for the food but has not encountered the food in a hungry state.

The above claims collectively confront us with a challenge. How are non-accidental matches[2] between preferences and primary motivational states possible?

This question is an interface problem.[3]

This interface problem was raised by Dickinson and Balleine:

‘we should search in vain among the literature for a consensus about the psychological processes by which primary motivational states, such as hunger and thirst, regulate simple goal-directed [i.e. instrumental] acts’ (Dickinson & Balleine, 1994, p. 1)

Background: Outcome-Driven vs Stimulus-Driven

Distinguish two kinds of action-guiding processes:

  1. Outcome-driven processes are processes guided by expectations concerning how likely an action is to bring about an outcome.
    They typically depend on action—outcome links, for example, lever-press—obtain-food. One example is goal-directed processes.

  2. Stimulus-driven processes are guided by the presence or absence of a stimlus. They typically depend on stimulus—stimulus and stimulus—action links, for example, magazine—food-smell and food-smell—eat. Examples include reflexes and habitual processes.

Warning. I could not find exactly this distinction in any of the textbooks or key sources I checked. Balleine (1992, p. 248) mentions ‘Pavlovian processes’ but does not say what they are. Domjan (2010, p. 209) comments:

‘Motivational processes in instrumental behavior have been addressed from two radically different perspectives and intellectual traditions, the associationist perspective rooted in Thorndike’s law of effect and Pavlovian conditioning, and the response-allocation perspective rooted in Skinner’s behavioral analysis. These two approaches differ in more ways than they are similar, making it difficult to imagine how they might be integrated. The fundamental concept in the associationist approach (the concept of an association) is entirely ignored in the response-allocation approach. Also, the mechanism of response allocation characterized by behavioral economics has no corresponding structure in the associationist approach. Both approaches have contributed significantly to our understanding of the motivation of instrumental behavior. Therefore, neither approach can be ignored in favor of the other.’ (Domjan, 2010, p. 209)

My proposal, which I take to be implicit in Dickinson and Balleine’s work, is twofold. First, outcome-driven and stimulus-driven processes are largely independent of each other. Second, primary motivational states directly influence only stimulus-driven processes.

Background: Classical vs Operant Conditioning

In operant conditioning, subjects are rewarded when they perform an action in response to a stimulus. (For example, when they press a lever in response to being in a particular cage.) Through operant conditioning, the subject may acquire a habit (stimulus—action link) or learn about the outcomes of actions (action—outcome link).[4]

We encountered operant conditioning in discussing evidence for the existence and influence of habitual processes (see Goal-Directed and Habitual Processes).

In classical conditioning, subjects are exposed to contingencies between stimuli. In Pavlov’s famous experiment, a bell was rung each time food appeared. Through classical conditining the subject learns about contingencies between stimuli (stimulus—stimulus link, e.g. bell—food).

On the importance of classical conditioning in human life:

Classical conditioning is used to study the ‘associative learning mechanisms [which] have been shaped by evolution to enable animals to store information about real causal relationships in their environment.’ (Dickinson, 1980, p. 26)

‘Classical conditioning is the process whereby we learn to predict when and what we might eat, when we are likely to face danger, and when we are likely to be safe. It is also integrally involved in the learning of new emotional reactions (e.g., fear or pleasure) to stimuli that have become associated with a significant event.’ (Domjan, 2010, p. 60)

Significance

Why does the independence of preferences from primary motivational states matter?

In What Are Preferences? we considered three attempts to anchor the notion of preference or desire. None of those attempts can straightforwardly accommodate the idea that there are two (or more) systems of motivational state.

In Dual Process Theory Opposes Decision Theory? we considered how to make the dual-process theory of instrumental action consistent with Jeffrey’s idea that decision theory can anchor a shared understanding of preference (and subjective probability). One possibility was to think of decision theory as characterising goal-directed processes only. But, as we noted back then, this would require that the rewards needed to characterise habitual processes are not rewards in virtue of your preferences. This is worth reconsidering if there are in fact tow kinds of motivational state which have distinct roles in explaining behaviour.

Background on Aversion

‘Pavlovian conditioning can lead to the learning of food preferences and aversions. A taste preference is learned if a flavor is paired with nutritional repletion or other positive consequences (e.g., Capaldi, Hunter, & Lyn, 1997). In contrast, a conditioned taste aversion is learned if ingestion of a novel flavor is followed by an aversive consequence such as indigestion or food poisoning. [...]

‘A growing body of evidence indicates that many human taste aversions are also the result of Pavlovian conditioning (Scalera, 2002).[5] [...] The typical aversion learning experience involves eating a distinctively flavored food and then getting sick. [...]

‘food aversion learning can be independent of rational thought processes and can go against a person’s own conclusions about the causes of the illness.’ (Domjan, 2010, p. 71)

Fun Fact

‘Classical conditioning was also independently discovered by Edwin Twitmyer in a Ph.D. dissertation submitted to the University of Pennsylvania in 1902 (see Twitmyer, 1974). Twitmyer repeatedly tested the knee-jerk reflex of college students by sounding a bell .5 seconds before hitting the patellar tendon just below the knee cap. After several trials of this sort, the bell was sufficient to elicit the knee-jerk reflex in some of the students.’ (Domjan, 2010, p. 60)

Ask a Question

Your question will normally be answered in the question session of the next lecture.

More information about asking questions.

Glossary

anchor : A theory, fact or other thing that is used by a group of researchers to ensure that they have a shared understanding of a phenomenon. An anchor is needed when it is unclear whether different researchers are offering incompatible claims about a single phenomenon or compatible claims about distinct phenomena. For example, we might take decision theory to anchor a shared understanding of belief and desire.
classical conditioning : Also called ‘Pavlovian conditioning’. A stimulus—stimulus link is learned through explosure to contingencies, causing you to respond to the first stimulus as if the second were present. Pavlov famously conditioned a dog to salivate on hearing a bell by having the bell sound before food arrived. Subject to effects like blocking and overshadowing.
decision theory : I use ‘decision theory’ for the theory elaborated by Jeffrey (1983). Variants are variously called ‘expected utility theory’ (Hargreaves-Heap & Varoufakis, 2004), ‘revealed preference theory’ (Sen, 1973) and ‘the theory of rational choice’ (Sugden, 1991). As the differences between variants are not important for our purposes, the term can be used for any of core formal parts of the standard approaches based on Ramsey (1931) and Savage (1972).
dual-process theory of instrumental action : Instrumental action ‘is controlled by two dissociable processes: a goal-directed and an habitual process’ (Dickinson, 2016, p. 177). (See instrumental action.)
goal-directed process : A process which involves ‘a representation of the causal relationship between the action and outcome and a representation of the current incentive value, or utility, of the outcome’ and which influences an action ‘in a way that rationalizes the action as instrumental for attaining the goal’ (Dickinson, 2016, p. 177).
habitual process : A process underpinning some instrumental actions which obeys Thorndyke’s Law of Effect: ‘The presenta­tion of an effective [=rewarding] outcome following an action [...] rein­forces a connection between the stimuli present when the action is per­formed and the action itself so that subsequent presentations of these stimuli elicit the [...] action as a response’ (Dickinson, 1994, p. 48). (Interesting complication which you can safely ignore: there is probably much more to say about under what conditions the stimulus–action connection is strengthened; e.g. Thrailkill, Trask, Vidal, Alcalá, & Bouton, 2018.)
inferential integration : For states to be inferentially integrated means that: (a) they can come to be nonaccidentally related in ways that are approximately rational thanks to processes of inference and practical reasoning; and (b) in the absence of obstacles such as time pressure, distraction, motivations to be irrational, self-deception or exhaustion, approximately rational harmony will characteristically be maintained among those states that are currently active.
interface problem : An interface problem may arise when two kinds of representation sometimes non-accidentally match: the problem is to explain how such matches are possible.
motor representation : The kind of representation characteristically involved in preparing, performing and monitoring sequences of small-scale actions such as grasping, transporting and placing an object. They represent actual, possible, imagined or observed actions and their effects.
operant conditioning : Also called ‘instrumental conditioning’. A stimulus—action link is learned through your action being rewarded when it occurs with the stimulus.
primary motivational state : A state such as hunger, thirst, satiety, aversion or sexual arousal. Primary motivational states are closely linked to biological needs. They are not all acquired through learning; and learning has limited effects on them, although classical conditioning can modify them (Capaldi et al., 1997).

References

Balleine, B. (1992). Instrumental performance following a shift in primary motivation depends on incentive learning. Journal of Experimental Psychology: Animal Behavior Processes, 18, 236–250. https://doi.org/10.1037/0097-7403.18.3.236
Balleine, B., & Dickinson, A. (1991). Instrumental performance following reinforcer devaluation depends upon incentive learning. The Quarterly Journal of Experimental Psychology Section B, 43(3), 279–296. https://doi.org/10.1080/14640749108401271
Bouton, M. E. (2016). Learning and Behavior: A Contemporary Synthesis (Second Edition, Second Edition). Oxford, New York: Oxford University Press.
Capaldi, E. D., Hunter, M. J., & Lyn, S. A. (1997). Conditioning with taste as the CS in conditioned flavor preference learning. Animal Learning & Behavior, 25(4), 427–436. https://doi.org/10.3758/BF03209849
Dickinson, A. (1980). Contemporary animal learning theory. Cambridge: Cambridge University Press.
Dickinson, A. (1994). Instrumental conditioning. In N. Mackintosh (Ed.), Animal learning and cognition. London: Academic Press.
Dickinson, A. (2016). Instrumental conditioning revisited: Updating dual-process theory. In J. B. Trobalon & V. D. Chamizo (Eds.), Associative learning and cognition (Vol. 51, pp. 177–195). Edicions Universitat Barcelona.
Dickinson, A., & Balleine, B. (1993). Actions and responses: The dual psychology of behaviour. In N. Eilan, R. McCarthy, & B. Brewer (Eds.), Spatial representation: Problems in philosophy and psychology (pp. 277–293). Oxford: Oxford University Press.
Dickinson, A., & Balleine, B. (1994). Motivational control of goal-directed action. Animal Learning & Behavior, 22(1), 1–18. https://doi.org/10.3758/BF03199951
Dickinson, A., & Balleine, B. (1995). Motivational Control of Instrumental Action. Current Directions in Psychological Science, 4(5), 162–167. https://doi.org/10.1111/1467-8721.ep11512272
Domjan, M. (2010). The Principles of Learning and Behavior. Wadsworth Cenage Learning.
Hargreaves-Heap, S., & Varoufakis, Y. (2004). Game theory: A critical introduction. London: Routledge. Retrieved from http://webcat.warwick.ac.uk/record=b2587142~S1
Jeffrey, R. C. (1983). The logic of decision, second edition. Chicago: University of Chicago Press.
Ramsey, F. (1931). Truth and probability. In R. Braithwaite (Ed.), The foundations of mathematics and other logical essays. London: Routledge.
Savage, L. J. (1972). The foundations of statistics (2nd rev. ed). New York: Dover Publications.
Scalera, G. (2002). Effects of Conditioned Food Aversions on Nutritional Behavior in Humans. Nutritional Neuroscience, 5(3), 159–188. https://doi.org/10.1080/10284150290013059
Sen, A. (1973). Behaviour and the Concept of Preference. Economica, 40(159), 241–259. https://doi.org/10.2307/2552796
Shettleworth, S. J. (1975). Reinforcement and the organization of behavior in golden hamsters: Hunger, environment, and food reinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 1, 56–87. https://doi.org/10.1037/0097-7403.1.1.56
Sugden, R. (1991). Rational Choice: A Survey of Contributions from Economics and Philosophy. The Economic Journal, 101(407), 751–785. https://doi.org/10.2307/2233854
Thrailkill, E. A., Trask, S., Vidal, P., Alcalá, J. A., & Bouton, M. E. (2018). Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior. Journal of Experimental Psychology: Animal Learning and Cognition, 44, 370–384. https://doi.org/10.1037/xan0000188

Endnotes

  1. Compare Dickinson & Balleine (1995, p. 164): ‘a shift in primary motivational state can have little or no direct impact on instrumental performance and thus, by implication, on the incentive value of the outcome. Rather, animals have to learn through experience with a particular food in the undeprived state that it has a low incentive value when they are not hungry.’ ↩︎

  2. Two motivational states match in a particular context just if, in that context, the actions one would cause and the actions the other would cause are not too different. ↩︎

  3. We encountered a different interface problem involving motor representations and intentions in The Interface Problem: Motor Representation vs Intention. ↩︎

  4. See Bouton (2016, pp. 28--9) for a simple guide (perhaps too simple). ↩︎

  5. Actually the source cited stresses that there are few human studies. ‘The CTA [conditioned taste aversion] has been extensively investigated in a wide variety of laboratory and wild animal species but only incidentally in humans’ (Scalera, 2002, p. 168). ↩︎