Larry Stein's research contributions in neuropsychopharmacology
Positive Reinforcement Stein’s contributions on the neurochemistry of positive reinforcement followed the discovery of brain self-stimulation by Olds and Milner in 1954 (1). Olds and co-workers had shown that the catecholamine antagonist, chlorpromazine, selectively suppressed self-stimulation behavior at doses which do not depress spontaneous motor activity. But –paradoxically-- Olds found at the same time that the catecholamine facilitator, amphetamine, also suppressed self-stimulation (because Olds’ amphetamine doses always were too high). By use of lower and more appropriate doses of amphetamine, Stein (2, 3) discovered that this catecholamine-enhancing drug will reliably, and often strikingly, increase the rate of self-stimulation. Furthermore, he was able to demonstrate for the first time that amphetamine facilitates self-stimulation behavior primarily by lowering the threshold current intensity for brain-stimulation reinforcement (3). Just at this point in the early 1960s, Hillärp’s group at the Karolinska Institute reported that most dopamine and norepinephrine fibers ascend in the brain in the lateral hypothalamic medial forebrain bundle. Since medial forebrain bundle electrodes had long been known from Olds’ work to support the very highest rates of self-stimulation, Stein put these anatomical facts together with his rudimentary but critical pharmacological observations i.e., amphetamine-induced facilitation and chlorpromazine-induced suppression of self-stimulation. Taken together, these facts led Stein (4, 5) to propose a novel hypothesis: that behavioral reinforcement is specialized neurochemically and is mediated by brain catecholamine systems. Stein’s original catecholamine hypothesis of positive reinforcement has guided research in this field for more than 50 years. It has led fruitfully to more precise hypotheses that dopamine, norepinephrine, opioid peptide, and cannabinoid systems may serve as important brain substrates for the reinforcement of operant behavior in animals and man. These ideas are now widely accepted, and they are supported by a vast number of brain self-stimulation, drug self-administration, electrophysiological, and optogenetic experiments from many laboratories all over the world, e.g. 6.
Central Mechanism of Amphetamine Intrigued by the idea that the brain’s reinforcement system might be selectively facilitated by amphetamine, Stein (7) next investigated the pharmacological mechanisms underlying amphetamine’s behavioral actions. When these studies began, two opposing hypotheses of amphetamine’s central actions were under consideration: #1) direct activation of brain catecholamine receptors, and #2) indirect activation of brain catecholamine receptors by drug-induced release of catecholamines. Many early studies had already indicated that amphetamine’s behavioral effects --unlike its peripheral effects-- are unaffected (or even are potentiated (8) by reserpine. These initially unexpected observations supported the receptor activation hypothesis (#1), because reserpine depletes brain catecholamines and leaves little endogenous transmitter in neuronal stores for amphetamine to release. Thus, for amphetamine’s behavioral actions, the release hypothesis (#2) had little currency in 1964. Peter Dews had invited Stein to discuss his self-stimulation research in a high-visibility FASEB Symposium (8). Stein’s original idea was to present some initial self-stimulation results which again suggested that amphetamine’s effects are undiminished after reserpine, indeed, he intended to support the receptor activation hypothesis (#1). But, in further experiments made shortly before the Dews’ symposium, Stein prudently assessed the effects of very large reserpine doses to ensure the near-total depletion of catecholamine stores. To his surprise, amphetamine’s effect on self-stimulation was now radically diminished in every rat tested (8). (At present, it is known that amphetamine’s behavioral effects are generally unimpaired even after severe, but incomplete, depletion of catecholamines). The quantitative precision of the operant methodology encouraged Stein to conclude from his unanticipated observations in 1964 that amphetamine must exert its behavioral actions indirectly via the release of catecholamines. These results and conclusions have since been amply confirmed (9).
Which Catecholamine is Most Reinforcing? For Stein, the question of which catecholamine is released by amphetamine (or positive brain stimulation) to produce behavioral reinforcement remained open in 1964: Dopamine and norepinephrine are the most likely candidates, but epinephrine cannot be definitely excluded. Arguments favoring norepinephrine are: a) it is the substance released by amphetamine from sympathetic nerves in the periphery, and b) it occurs in high concentrations throughout the reward system. Arguments favoring dopamine are: a) it is a much better substrate for monoamine oxidase than norepinephrine, and b) it occurs in high concentrations in the extrapyramidal system, the final common pathway, perhaps, for the modulation of behavior by reinforcing stimuli (7, p.848). Unfortunately, good pharmacological tools for the absolute discrimination of dopamine’s and norepinephrine’s behavioral actions were not available in this period. Self-stimulation data from Stein’s laboratory later revealed behavioral facilitation after intraventricular administrations of l-norepinephrine, but not of d-norepinephrine or dopamine (10). Such findings led Stein initially to favor norepinephrine over dopamine as the most salient mediator of amphetamine-induced and self-stimulation reinforcement, but strong evidence favoring dopamine was rapidly accumulating (e.g., 11). In 1974, Stein realized that a stringent test of the dopamine hypothesis might be achieved by offering apomorphine (a powerful D2-dopamine receptor agonist) as reinforcement in drug self-administration tests. Small doses of apomorphine induce nausea and vomiting in many animal species, including man—thus, if despite its potential adverse actions, apomorphine could be shown to reinforce self-administration behavior, the dopamine hypothesis would be strongly supported. In collaboration with Bruce Baxter and others at Wyeth responsible for the company’s ongoing self-administration screening activity, Stein found for the first time that rats will avidly self-administer apomorphine (12). In 1992, Stein and his student David Self further demonstrated that rats will self-administer the D1 dopamine receptor agonist SKF 38,393 as reliably as they will self-administer D2-dopamine agonists (13). This observation refuted the view, widely-held at the time, that D1-dopamine receptors are not involved in positive reinforcement.
Punishment Stein’s important studies of punishment system neurochemistry and the mechanism of action of benzodiazepines were initiated in 1970. Graeff and Schoenfeld (14) had just reported very large increases in the punished response rates of pigeons after injections of serotonin antagonists. At about the same time, biochemical studies in many laboratories were revealing that benzodiazepines and related agents (barbiturates, meprobamate) substantially reduce the turnover of serotonin and norepinephrine in the brain. Previously, in 1968, Margules and Stein (15) had found that the punishment-lessening and response-depressant effects of oxazepam followed opposite courses in the Geller-Seifter “conflict” test. In confirmation of clinical observations, oxazepam’s depressant action (measured by a decrease of nonpunished behavior) was observed to undergo tolerance after 3 to 6 doses, while its “antianxiety” action (measured by an increase in the rates of punished behavior) failed to show tolerance and often even increased after repeated drug administrations.
In 1972, Stein’s group (16) integrated these behavioral and biochemical observations and reported that oxazepam’s effects on norepinephrine turnover rapidly undergoes tolerance, whereas its effects on serotonin turnover remains stable. This led Stein and his co-workers to hypothesize that benzodiazepines may exert their punishment-lessening effects by a reduction in central serotonin activity (and their response-depressant effects by a reduction in central norepinephrine activity). Stein’s group (17) also provided the first behavioral evidence (i.e., selective picrotoxin reversal of the punishment-lessening effects of benzodiapines) in support of the now well-established view that benzodiazepines may influence monoamine turnover by secondary effects, which arise from a primary action on the GABAA receptor complex. (Picrotoxin is a noncompetitive antagonist for GABAA chloride channels). The serotonin-GABAA hypothesis of benzodiazepine’s behavioral actions has since been amply confirmed (e.g., 18).
Opioid Peptides The first identification in brain of natural opiate receptor agonists—met-enkephalin and leu-enkephalin—impacted pharmacology and neuroscience in 1975 with a force similar to that exerted on physics by the discovery of radioactivity at the end of the 19th century. For months, neuropharmacologists were engrossed with little else, and Stein was no exception. Since enkephalins are pentapeptides, with poor brain penetration, it was necessary to inject them directly into the brain to study their behavioral actions. In 1976-77, Stein’s group reported two studies in Nature (19, 20) which demonstrated for the first time that: 1) “methionine-enkephalin and leucine-enkephalin, when administered through permanently-indwelling cannulae in the lateral ventricles of rats, induce a profound analgesia in vivo that is fully reversible by naloxone.” (19, p. 625) and 2) “that rats will work for enkephalin injections delivered directly into the ventricles of their own brains. Of particular interest, in the light of the generally superior potency of the methionine peptide, is the observation that leucine-enkephalin is self-administered with greater avidity than methionine-enkephalin.” (20, p. 556). Perhaps unsurprisingly, these papers on enkephalin—as well as an anatomical mapping study (21) on the distribution of enkephalin-immunoreactive cell bodies in the rat brain performed with Tomas Hökfelt and others—are among the most frequently cited articles in Stein’s bibliography.
A Cellular Analog of Operant Conditioning Stein’s most recent work concerns the most difficult question that he has addressed over a long career: Which brain structures are the most likely to receive signals from (and thus be reinforced by) the dopamine and opioid-peptide reinforcement systems. Since it is chiefly operant behavior that is reinforced, it is plausible to assume that behavior-controlling brain substrates are major targets of the reinforcing systems. But little or nothing is known about the actual neural substrates of operant behavior.
It is commonly believed that positive reinforcement is exerted at the systems level and probably involves the strengthening or reorganization of complex “whole-response” circuitries. But whether or not particular behavioral variations are treated as the same or as different responses depends on the reinforcement contingencies. Thus, for example, if lever-press responses of 5-g and 10-g force are reinforced indiscriminately, both are counted as the same “correct” response; however, if they are selectively reinforced, the behavioral variations clearly are regarded as different responses. The fact that closely similar behavioral variations may be reinforced either indiscriminately or selectively is presumptive evidence that the unit of reinforcement cannot be the whole response itself. B.F. Skinner (22) has deeply considered these problems and has concluded that the operant response must be a composite made up of elements, and that it is these “response elements” or “behavioral atoms” which are the more likely functional units for reinforcement.
In a first and very preliminary approach, Stein has assumed that Skinner’s behavioral atoms might conceivably be represented at the neuronal level by individual cellular responses. Using a brain slice preparation, he has attempted to demonstrate --in the absence of most of the brain-- the in vitro operant conditioning of hippocampal bursting activity using micropressure applications of dopamine, cocaine, or other behaviorally-reinforcing agents directly to the cell body as cellular reinforcements. The in vitro reinforcement test employs training procedures closely analogous to those of behavioral operant conditioning. The most important, and indeed defining, feature of behavioral operant conditioning is an absolute requirement for response-reinforcement contingency. Accordingly, in Stein’s cellular analog, it was obligatory to show that only burst-contingent (and not burst-independent) microapplications of reinforcing drugs will produce significant enhancement of hippocampal bursting.
Stein and his collaborators (23, 24) have observed that the bursting responses of individual CA1 pyramidal neurons were progressively increased in a dose-related manner by burst-contingent micropuffs of dopamine and cocaine, whereas the bursting responses of CA3 units similarly were increased by morphine and dynorphin. The same microinjections, administered independently of cellular bursting, failed to facilitate and frequently suppressed CA1 and CA3 bursting, respectively—this observation suggested that nonspecific stimulation of cellular activity is an unlikely explanation of the facilitatory action of the burst-contingent injections. Experiments with glutamate, an excitatory transmitter that is not commonly associated with the behaviorally-reinforcing effects of drugs, also contradicted the nonspecific-stimulation hypothesis. Burst-contingent injections of glutamate always failed to increase CA1 bursting.
All of the above observations are consistent with the possibility that the bursting of hippocampal cells may be “operantly conditioned in vitro” by burst-contingent microinjections of behaviorally-reinforcing transmitters or drugs. Stein et al. (1994) conclude: “If so, and if we have actually managed to put ‘a single neuron in a Skinner box’ …we may have had the good luck to get a glimpse of a Skinnerian behavioral atom.” (24, p. 167).
Summary Over a long career,Stein has made several important experimental and theoretical contributions to behavioral pharmacology. Especially noteworthy are his pioneering investigations on the catecholamine mediation of operant reinforcement, serotonergic mediation of punishment, indirect (catecholamine-releasing) central actions of amphetamine, and first demonstration of the analgesic and reinforcing effects of enkephalins. His published work has had high impact on behavioral pharmacology—as one indication, among his peer-reviewed papers are 17 Science and Nature articles. In particular, Stein’s catecholamine hypothesis of positive reinforcement has guided research on behavioral and stimulant drug reinforcement for 50 tears.
References
1. J. Olds and P. Milner (1954). J. comp. physiol. Psychol. 47, 419-427.
2. L. Stein (1964). In: H. Steinberg, A.V.S. deReuck, and J. Knight (Eds.). Animal behavior and drugaction. J. & A. Churchill, Ltd., London, pp .91-118.
3. L. Stein and O.S. Ray (1960). Psychopharmacologia (Berl.)1, 251-256.
4. L. Stein (1962). Effects and interactions of imipramine, chlorpromazine, reserpine and amphetamine on self-stimulation: possible neurophysiological basis of depression. In Wortis, J (ed.) Recent Advances in Biological Psychiatry, pp.288-308, New York: Plenum Press.
5. L. Stein (1968). Chemistry of reward and punishment. In Efron, DH (ed.) Proceedings of the American College of Neuropschopharmacology, pp 105-123. Washington, DC: US Government Printing Office.
6. R.A. Wise (2008). Dopamine and reward: the anhedonia hypothesis 30 years on. Neurotox. Res. 14 (2-3), 169-183.
7. L. Stein (1964). Federation Proceedings23 No. 4, 836-850.
8. C.B. Smith (1963). J. Pharmacol. Exptl. Therap. 142, 343-350.
9. E. Costa and S. Garattini (Eds.) (1970). Amphetamines and related compounds. Raven Press, New York.
10. C.D. Wise, B.D. Berger, and L. Stein (1973). Biological Psychiatry6, 3-21.
11. R.A. Wise (1982). Neuroleptics and operant behavior: the anhedonia hypothesis. Behav. Brain Sci. 5, 39-87.
12. B. Baxter, M.I. Gluckman, L. Stein, and R.A Scerni (1974. Pharm.Bioch. Behav. 2, 387-392.
13. D. Self and L. Stein (1992). Brain Research582, 349-352).
14. F.G. Graeff and R.I. Schoenfeld (1970). J. Pharmacol. Exptl. Therap.173, 277-283.
15. D.L. Margules and L. Stein (1968). Psychopharmacologia (Berl.)13, 74-80.
16. C.D. Wise, B.D. Berger, and L. Stein (1972). Science177, 180-183.
17. L. Stein, C.D. Wise and J.D. Belluzzi (1975). In E. Costa and P. Greengard (Eds.). Mechanism of action of benzodiazepines. Raven Press, New York.
18. J. Sepinwall and L. Cook (1980). Federation Proceedings39 No. 12, 3024-3031
19. J.D. Belluzzi et al. (1976) Nature260, 625-626.
20. J.D. Belluzzi and L. Stein (1977) Nature266, 556-558.
21. T. Hökfelt, R. Elde, O. Johansson, L. Terenius, and L. Stein (1977). Neuroscience Letters 5, 25-31.
22. B.F. Skinner (1953). Science and human behavior. Free Press, New York.
23. L. Stein, B.G. Xue and J.D. Belluzzi (1993). J. Exptl. Anal. Behav. 60, 41-53.
24. L. Stein, B.G. Xue and J.D. Belluzzi (1994). J. Exptl. Anal. Behav. 61, 155-168.
January 5, 2016