Preference vs Aversion: A Dissociation
Your preferences can be incompatible with your aversions (and thereby with primary motivational states). This shows that there is not a single system of preferences in rats or humans.
This recording is also available on stream (no ads; search enabled). Or you can view just the slides (no audio or video). You should not watch the recording this year, it’s all happening live (advice).
If the slides are not working, or you prefer them full screen, please try this link.
There are at least two kinds of motivational state, which have distinct roles in explaining behaviour.
One is desires or preferences. These are states that can be influenced by fashion and all kinds of learning.
The other is primary motivational states. These include hunger, thirst, satiety, aversion and disgust. They are linked to biological needs and not always learned. Although they can be modified by learning, there are limits on the influence of learning.
How are primary motivational states and desires related?
Consider two views:
Wrong View Primary motivational states and preferences form a single system for guiding action selection. Unless something goes badly wrong, your primary motivational states inform your preferences directly. To illustrate, if you are hungry for a food, you desire it; and if you are averse to a food, you do not desire it.
In this section we consider evidence against the Wrong View and in favour of:
Correct View There are at least two kinds of motivational state which have distinct roles in explaining behaviour. If primary motivational states inform your preferences at all, they do so only indirectly. You may hunger for a food you do not desire to eat, and you may desire to eat a food to which you are averse.
Anecdote: Dickinson’s Water Melons
Dickinson ate melons for the first time. Shortly after consumed red wine and suffered mild toxicosis. Waking up the next day, Dickinson felt thirsty.
‘once again in need of refreshment, he readily retraced the route to the water melon stall. But when confronted with the direct signal for approach, the sight of those sliced, juicy, rosy-red segments which had looked and were so refreshing on the previous occasion, his appetite abated. And indeed, when he managed to take a bite, he discovered that he had an aversion to water melon with the consequence that none has passed his lips since that day.’ (Dickinson & Balleine, 1993, p. 285)
The aversion makes sense because becoming unwell after eating a novel food can cause aversion to it (Domjan, 2010, p. 71).
Puzzle: if Dickinson was not already averse to melon when he work up, how could eating melon cause aversion? If Dickinson was already averse, why did he set out to eat melon?
Does the Correct View (see above) suggest a solution to this puzzle?
Can your primary motivational states diverge from your preferences?
Can hunger drive you to seek a novel food even tho you have no desire to eat it? And can satiety reign in your search for a food even though you desire to eat it?
Yes (Balleine, 1992)!
And can sugar solution rank highly among your preferences even after you have become averse to it?
Yes (Balleine & Dickinson, 1991)!
The evidence for positive answers to these questions supports the Correct View (see above): Preferences and primary motivational states have distinct roles in explaining behaviour. If primary motivational states inform your preferences at all, they do so only indirectly.
Conclusion: your behaviours are driven by two (or more) motivational systems which are, to an interesting degree, independent of each other.
This conclusion gives rise to an interface problem ...
An Interface Problem
We have seen evidence for these claims:
Primary motivational states guide some actions.
Preferences guide some actions.
Pursuing a single goal can involve both kinds of state, as, for instance, when the rat obtains food by pressing the lever and entering the magazine to retrieve it.
Primary motivational states can differ from preferences, as, for instance, when the rat is hungry for the food but has not encountered the food in a hungry state.
This interface problem was raised by Dickinson and Balleine:
‘we should search in vain among the literature for a consensus about the psychological processes by which primary motivational states, such as hunger and thirst, regulate simple goal-directed [i.e. instrumental] acts’ (Dickinson & Balleine, 1994, p. 1)
Background: Outcome-Driven vs Stimulus-Driven
Distinguish two kinds of action-guiding processes:
Outcome-driven processes are processes guided by expectations concerning how likely an action is to bring about an outcome. They typically depend on action—outcome links, for example, lever-press—obtain-food. One example is goal-directed processes.
Stimulus-driven processes are guided by the presence or absence of a stimlus. They typically depend on stimulus—stimulus and stimulus—action links, for example, magazine—food-smell and food-smell—eat. Examples include reflexes and habitual processes.
Warning. I could not find exactly this distinction in any of the textbooks or key sources I checked. Balleine (1992, p. 248) mentions ‘Pavlovian processes’ but does not say what they are. Domjan (2010, p. 209) comments:
‘Motivational processes in instrumental behavior have been addressed from two radically different perspectives and intellectual traditions, the associationist perspective rooted in Thorndike’s law of effect and Pavlovian conditioning, and the response-allocation perspective rooted in Skinner’s behavioral analysis. These two approaches differ in more ways than they are similar, making it difficult to imagine how they might be integrated. The fundamental concept in the associationist approach (the concept of an association) is entirely ignored in the response-allocation approach. Also, the mechanism of response allocation characterized by behavioral economics has no corresponding structure in the associationist approach. Both approaches have contributed significantly to our understanding of the motivation of instrumental behavior. Therefore, neither approach can be ignored in favor of the other.’ (Domjan, 2010, p. 209)
My proposal, which I take to be implicit in Dickinson and Balleine’s work, is twofold. First, outcome-driven and stimulus-driven processes are largely independent of each other. Second, primary motivational states directly influence only stimulus-driven processes.
Background: Classical vs Operant Conditioning
In operant conditioning, subjects are rewarded when they perform an action in response to a stimulus. (For example, when they press a lever in response to being in a particular cage.) Through operant conditioning, the subject may acquire a habit (stimulus—action link) or learn about the outcomes of actions (action—outcome link).
In classical conditioning, subjects are exposed to contingencies between stimuli. In Pavlov’s famous experiment, a bell was rung each time food appeared. Through classical conditining the subject learns about contingencies between stimuli (stimulus—stimulus link, e.g. bell—food).
On the importance of classical conditioning in human life:
Classical conditioning is used to study the ‘associative learning mechanisms [which] have been shaped by evolution to enable animals to store information about real causal relationships in their environment.’ (Dickinson, 1980, p. 26)
‘Classical conditioning is the process whereby we learn to predict when and what we might eat, when we are likely to face danger, and when we are likely to be safe. It is also integrally involved in the learning of new emotional reactions (e.g., fear or pleasure) to stimuli that have become associated with a significant event.’ (Domjan, 2010, p. 60)
Why does the independence of preferences from primary motivational states matter?
In What Are Preferences? we considered three attempts to anchor the notion of preference or desire. None of those attempts can straightforwardly accommodate the idea that there are two (or more) systems of motivational state.
In Dual Process Theory Opposes Decision Theory? we considered how to make the dual-process theory of instrumental action consistent with Jeffrey’s idea that decision theory can anchor a shared understanding of preference (and subjective probability). One possibility was to think of decision theory as characterising goal-directed processes only. But, as we noted back then, this would require that the rewards needed to characterise habitual processes are not rewards in virtue of your preferences. This is worth reconsidering if there are in fact tow kinds of motivational state which have distinct roles in explaining behaviour.
Background on Aversion
‘Pavlovian conditioning can lead to the learning of food preferences and aversions. A taste preference is learned if a flavor is paired with nutritional repletion or other positive consequences (e.g., Capaldi, Hunter, & Lyn, 1997). In contrast, a conditioned taste aversion is learned if ingestion of a novel flavor is followed by an aversive consequence such as indigestion or food poisoning. [...]
‘A growing body of evidence indicates that many human taste aversions are also the result of Pavlovian conditioning (Scalera, 2002). [...] The typical aversion learning experience involves eating a distinctively flavored food and then getting sick. [...]
‘food aversion learning can be independent of rational thought processes and can go against a person’s own conclusions about the causes of the illness.’ (Domjan, 2010, p. 71)
‘Classical conditioning was also independently discovered by Edwin Twitmyer in a Ph.D. dissertation submitted to the University of Pennsylvania in 1902 [...] Twitmyer repeatedly tested the knee-jerk reflex of college students by sounding a bell 0.5 seconds before hitting the patellar tendon just below the knee cap. After several trials of this sort, the bell was sufficient to elicit the knee-jerk reflex in some of the students.’ (Domjan, 2010, p. 60)
Ask a Question
Your question will normally be answered in the question session of the next lecture.
To illustrate, one way of matching is for the B-outcomes to be the A-outcomes. Another way of matching is for the B-outcomes to stand to the A-outcomes as elements of a more detailed plan stand to those of a less detailed one.
[of plan-like structures] In the simplest case, plan-like hierarchies of motor representations match if they are identical. More generally, plan-like hierarchies match if the differences between them do not matter in the following sense. For a plan-like hierarchy in an agent, let the self part be those motor representations concerning the agent's own actions and let the other part be the other motor representations. First consider what would happen if, for a particular agent, the other part of her plan-like hierarchy were as nearly identical to the self part (or parts) of the other's plan-like hierarchy (or others' plan-like hierarchies) as psychologically possible. Would the agent's self part be different? If not, let us say that any differences between her plan-like hierarchy and the other's (or others') are not relevant for her. Finally, if for some agents' plan-like hierarchies of motor representations the differences between them are not relevant for any of the agents, then let us say that the differences do not matter.
[of motivational states] Two motivational states match in a particular context just if, in that context, the actions one would cause and the actions the other would cause are all proper ways of fulfilling both motivational states.
Compare Dickinson & Balleine (1995, p. 164): ‘a shift in primary motivational state can have little or no direct impact on instrumental performance and thus, by implication, on the incentive value of the outcome. Rather, animals have to learn through experience with a particular food in the undeprived state that it has a low incentive value when they are not hungry.’ ↩︎
We encountered a different interface problem involving motor representations and intentions in The Interface Problem: Motor Representation vs Intention. ↩︎
Actually the source cited stresses that there are few human studies. ‘The CTA [conditioned taste aversion] has been extensively investigated in a wide variety of laboratory and wild animal species but only incidentally in humans’ (Scalera, 2002, p. 168). ↩︎