Brain Prize for Dayan, Dolan and Schultz
Three neuroscientists who work in the UK have on 6 March 2017 won the world’s most valuable prize for brain research. This year, The Brain Prize for 2017 is awarded to Peter Dayan, Ray Dolan and Wolfram Schultz for their analysis of how the brain recognises and processes reward. The capacity to link reward to events and actions is the foundation of human and animal survival, and problems with the processing of reward lie at the heart of many neurological and psychiatric disorders.
The Brain Prize, awarded by the Lundbeck Foundation in Denmark, is worth one million Euros. Awarded annually, it recognises one or more scientists who have distinguished themselves by an outstanding contribution to neuroscience.
The research of this year’s winners has far-reaching implications for understanding human behaviour, including decision-making, gambling, drug addiction, compulsive behaviour and schizophrenia.
Reward is essential to survival because humans and other animals need to learn to direct their decisions and their actions towards outcomes that will satisfy their needs, and away from danger. This means that they have to learn which events in the environment predict future rewards and punishments. For instance, if you feel hungry and see a building with a sign ‘restaurant’, you are likely to enter because the sign predicts that your hunger will be reduced if you go inside.
The sense of reward is surprisingly complicated. It is influenced and determined by many things, such as taste and smell, as well as by fundamental motivations such as hunger or thirst. In turn, it influences choices, decisions and even attention. Many regions of the brain process information associated with reward, but one central linchpin for the regulation of learning and performance is a neurotransmitter (chemical messenger) in the brain called dopamine.
Thirty years ago, German-born Wolfram Schultz, professor of neuroscience now at the University of Cambridge, was studying learning in monkeys at the University of Fribourg in Switzerland. He developed methods for recording activity from neurons (nerve cells) that use dopamine to transmit information to other neurons. He found that before learning, these dopamine neurons respond whenever a reward - fruit juice - is given to the monkey, but if the monkey is shown various visual patterns and has to respond to one of them in order to secure the reward, the pattern of response changes as the animal learns. The dopamine neurons now respond when the correct visual pattern appears, and the response to the reward itself disappears. If no reward is given, the activity of dopamine neurons actually decreases at the expected time after the visual signal; but if the reward is delivered at an unexpected time, the neurons respond to it.
“This is the biological process that makes us want to buy a bigger car or house, or be promoted at work,” said Schultz. Every time we get the reward, our dopamine neurons affect our behaviour. “They are like little devils in our brain that drive us towards more rewards.”
British computational neuroscientist, Peter Dayan, director of the Gatsby Computational Neuroscience Unit, University College London, is recognised internationally as a leader in the rapidly developing field of computational neuroscience. When working at the Salk Institute in California, Dayan realised that the pattern of activity of dopamine neurons described by Schultz corresponds to a signal known - from the earliest days of artificial intelligence - as a ‘reward prediction error’.
This signal is the difference between the reward that is actually delivered and the reward that is predicted to be delivered. Such a signal, continuously updated in the brain on the basis of previous reward, is perfectly positioned to control learning. If predictions are wrong, for instance being too pessimistic, reward prediction errors can adjust them upwards to be more optimistic. If the prediction error signals that something unexpectedly good follows a particular choice that we made, we become more likely to act or behave in the same way in the future.
Together, prediction errors sculpt our expectations and experience of the world.
“For example, imagine that you choose between restaurants based on predicting how good they are. Then, if the one you chose is better than expected, the positive prediction error allows you to update your prediction. Next time you are faced with a restaurant choice, you are more likely to pick the one that was better,” said Dayan.
This link between dopamine and prediction error was one of the spurs for an explosion of work using theoretical ideas and computational models to link artificial intelligence, economics, mathematics, engineering and statistics to swathes of results in psychology and neuroscience.
“Dopamine of course does not work alone,” Dayan noted. “Indeed, there is currently great interest in understanding the many cooperating and competing brain systems that contribute to both good and bad choices, and indeed manipulating them when things go wrong.”
Both Dayan and Schultz have made further important contributions to the field of learning, risk-taking and decision-making. Schultz is currently using experimental economics and economic choice theory to demonstrate in a stringent manner how humans and animals get the best possible reward in various situations that involve risk. He has also used brain imaging to study changes in reward-related signals in Parkinson’s patients, in smokers, and in people with drug addiction.
Dopamine neurons play a ‘devilish’ role in drug addiction. “Addictive drugs generate, hijack and amplify the reward signal and induce exaggerated and uncontrolled effects of dopamine on the brain,” Schultz explained.
Professor Ray Dolan was born in the Irish Republic is the director of the new Max Planck Centre for Computational Psychiatry and Ageing, and the Wellcome Centre for Neuroimaging at University College London. Dolan has been a leader in the development and use of methods for imaging the human brain, in order to understand the mechanisms of emotion, learning and decision-making.
Through his pioneering application of mathematical models to brain imaging and behaviour, together with his discoveries on the action of dopamine and other neurotransmitters, he has shown that the basic idea of a prediction error accounts for how humans learn about reward and punishment and also how we learn about the preferences of other people. He has used innovative ways to determine how dopamine affects brain function, including its impact on memory function and the disposition to take risks. This work helps researchers understand a range of psychiatric problems, for example behavioural impulsivity and apathy.
Dayan and Dolan have worked collaboratively over the past decade to probe how reward learning impacts on complex human questions, including motivational drive, variation in happiness, and a propensity towards gambling.
“One puzzling clinical problem is why some patients treated with drugs that boost dopamine function, for example in Parkinson’s disease, fall prey to pathological gambling. Our work has shown that this effect is, at least in part, due to dopamine amplifying an innate tendency to repeat activities that are rewarding,” said Dolan.
The winners gratefully acknowledged the contributions of their many colleagues and collaborators, as well as the institutions and funding agencies that have supported their work, especially the Gatsby Charitable Foundation, the Wellcome Trust, University College London, the University of Cambridge, the University of Fribourg, the Salk Institute and California Institute of Technology.
“I see it as a wonderful tribute to my superb collaborators, students and postdocs,” said Dayan.
“I am immensely gratified that the Lundbeck Foundation acknowledged the importance of this work for understanding psychiatric illness,” said Dolan.
“The Brain Prize is a fantastic reward for our research group. I can hear our dopamine neurons jumping up and down!” Schultz said.
Professor Sir Colin Blakemore (University of London), chairman of the Brain Prize selection committee said, “The judges concluded that the discoveries made by Wolfram Schultz, Peter Dayan and Ray Dolan were crucial for understanding how the brain detects reward and uses this information to guide behaviour. This work is a wonderful example of the creative power of interdisciplinary research, bringing together computational explanations of the role of activity in the monkey brain with advanced brain imaging in human beings to illuminate the way in which we use reward to regulate our choices and actions. The implications of these discoveries are extremely wide-ranging, in fields as diverse as economics, social science, drug addiction and psychiatry”.
The winners will share the prize of one million Euros, which will be presented to them at a ceremony on 4 May in Copenhagen by His Royal Highness Crown Prince Frederik of Denmark.
For further information contact
Elaine Snell, Snell Communications Ltd, London, UK
020 7738 0424 / 07973 953 794 email@example.com
NOTES TO EDITORS
• The one million Euro Brain Prize is awarded by the Lundbeck Foundation in Denmark. In 2017, the Prize will be awarded for the seventh consecutive year.
• The Brain Prize is a personal prize, awarded to one or more scientists who have achieved distinction through outstanding contributions to European brain research. More information can be found on The Brain Prize website, www.thebrainprize.org
• The Lundbeck Foundation is one of the largest, industrial foundations in Denmark with a market value of more than DKK 60 billion. The Lundbeck Foundation annually grants approx. DKK 500 million in support of Danish-based, biomedical research of the highest, international quality with a special focus on brain health as part of the foundation’s vision to create better lives through new knowledge.
Further background reading about the reward system:
Dopamine reward prediction error coding – W Schultz, 2016 Dialogues in Clinical Neuroscience
Goals and habits in the brain – R Dolan, P Dayan, 2013, Neuron
About the prizewinners / biographies
Peter Dayan is British. He studied mathematics at the University of Cambridge and did a PhD in Cognitive Science at the University of Edinburgh, focusing on statistical and neural network models of learning. He was a postdoctoral fellow at the Computational Neurobiology Laboratory at The Salk Institute, and the Department of Computer Science at the University of Toronto. He held an Assistant Professorship at MIT and moved back to the United Kingdom in 1998, to help found the Gatsby Computational Neuroscience Unit, University College London. He became Director in 2002, as well as a Professor of Computational Neuroscience. He won the Rumelhart Prize in 2012.
Raymond Dolan, born in the Republic of Ireland, studied medicine at University College Galway, National University of Ireland. He completed his specialist training in psychiatry in the UK. In 1994 he moved to the Institute of Neurology, University College London. He was a founding member of the Wellcome Functional Imaging Laboratory (FIL). In 2006, he became founding Director of the Wellcome Trust Centre for Neuroimaging at UCL. He took on the role of Director of a new UCL-Max Planck Centre for Computational Psychiatry and Ageing Research. He is a Fellow of the Royal College of Physicians, the Royal College of Psychiatrists, the Academy of Medical Sciences and the Royal Society. He won the Minerva Foundation Golden Brain Award (2006), the International Max Planck Research Award (2007), and the Zülch Prize (2013) (shared with Wolfram Schultz).
Wolfram Schultz has lived and worked in Germany, Switzerland and Britain. He studied medicine, mathematics and philosophy at the Universities of Hamburg and Heidelberg, Germany (1966-73), and did his postdoctoral training in Göttingen, Germany (1973-75), the State University of New York, Buffalo (1975-6), and the Karolinska Institute, Stockholm, Sweden (1976-77). He then worked in the Institute of Physiology at the University of Fribourg, Switzerland, from 1977 to 2001, when he moved to the University of Cambridge as Professor of Neuroscience and Wellcome Trust Principal Research Fellow. He is also a Visiting Research Associate at the California Institute of Technology in Pasadena. Schultz has won a number of awards, including
the Minerva Foundation Golden Brain Award (2002), and the Ipsen Prize (2005) for Neuronal Plasticity. He is a Fellow of the Royal Society.