To elucidate complex brain function, a computational approach is widely accepted in system neuroscience as well as clinical disciplines. In particular, computational models of neuromodulators, such as dopamine and serotonin, are indispensable to make clear the neural system of prediction and decision- making, and these models have been frequently tested using an experimental approach. Prediction error signal in reinforcement learning theory is a representative computational model for the role of dopamine in prediction and decision- making. This model was proposed on the basis of electrophysiological data from a series of studies on dopaminergic neurons in monkeys conducted by Schultz et al. in the 1990s. In classical conditioning experiment, dopaminergic neurons in monkeys responded to rewards before learning, whereas after the learning tasks, these neurons started to respond to the conditioned stimuli. This neuronal alteration observed in dopaminergic neurons was revealed to be similar to the prediction error signal in reinforcement learning. Based on this new discovery of the role of dopaminergic neurons, reinforcement learning model mediated by the cortico- basal ganglia circuit has been proposed, and this model is supported by studies using electrophysiological techniques and functional magnetic resonance imaging of human brain. The representative role of serotonin in prediction and decision- making may be “impulsive choice” behavior. Impulsive choice is defined as a behavioral preference of immediate small rewards over distant large rewards, and rats developed impulsive choice behavior when the serotonergic system in the brain was destroyed. On the basis of these findings, it has been proposed that serotonin is involved in delay discounting. However, because serotonergic neurons have a wide range of projection and a large number of serotonin receptor subtypes exist, many questions remain about the functional role of serotonin in impulsive choice. Consequently, various computational models of serotonin, including temporal discounting, have been proposed and are being investigated.

View full abstract

For survive in the ever-changing natural environment, it is essential to assign long-term reward value for actions. Although midbrain dopamine neurons are known to signal reward value and its prediction error, it is not examined experimentally whether and how dopamine neurons encode long- term value of multiple future rewards (TD error), as suggested in reinforcement learning theories. We address this issue by studying 185 dopamine neuron activities recorded from three monkeys that performed a multi- step choice task for three rewards. In the task, they explored a reward among three alternatives and then exploited this knowledge to receive two additional rewards by repeating the same choice in subsequent trials. Dopamine responses to the start cues represented expectations of multiple future rewards; the sum of immediate and discounted future rewards. In accordance with this result, responses to the reinforcers beeps reflected the errors of the multiple future rewards. These responses were quantitatively predicted by theoretical descriptions of the value function with time discounting in reinforcement learning. Moreover, we confirmed that these responses were established through learning the multistep choice paradigm for rewards. These findings demonstrate that dopamine neurons “learn” to encode the long-term value of multiple future rewards with distant rewards discounted.

View full abstract

Classic theories suggest that central serotonergic neurons are involved in the behavioral inhibition that is associated with the prediction of negative rewards or punishment. Failed behavioral inhibition can cause impulsive behaviors. However, the behavioral inhibition that results from predicting punishment is not sufficient to explain some forms of impulsive behavior. Recently, we have found that serotonergic neurons increase their tonic firing rate when rats await food and water rewards and conditioned reinforcer tones. The rate of tonic firing during the delay period was significantly higher when rats were waiting for rewards than for tones, and rats were unable to wait as long for tones as for rewards. These results suggest that increased serotonergic neuronal firing facilitates waiting behavior when there is the prospect of a forthcoming reward and that serotonergic activation contributes to the patience that allows rats to wait longer. In this article, we propose that the forebrain serotonergic system is involved in “waiting to avoid punishment” for future punishments and “waiting to obtain reward” for future rewards.

View full abstract

Humans and animals prefer immediate over delayed rewards (delay discounting). This preference for smaller-but-sooner over larger-but-later rewards shows substantial interindividual variability and has been considered as an index of impulsivity. Previous studies have reported that decreased central serotonin levels or lesions of specific parts of cortico- striatal circuitry (e. g. ventral striatum) causes steep rate of delay discounting and consequent impulsive action selection. Recently, Tanaka et al. (2007) found that a graded map of delay discount rate in the striatum and insula using the functional magnetic resonance imaging (fMRI) ; activities in the ventral striatum and anterior insula were correlated with impulsive reward prediction, while those in the dorsal caudate and posterior insula were correlated with deliberate reward prediction. In addition, they showed that the correlated activity in the ventral striatum was dominant at low central serotonin level. These results facilitated our understanding of neural substrates of impulsivity. Although cumulative evidence has reported dysfunction in serotonergic system or cortico-striatal circuitry in various psychiatric disorders, the relationships between clinical symptoms and these biological backgrounds remain unclear. We suggest the possibility to construct the comprehensive model through the serotonergic dysfunction and related alteration of cotrico-striatal circuitry and impulsive action selection.

View full abstract

[in Japanese]

View full abstract

Experience as well as genetic programs affect brain development. While most studies of experience-dependent effects have focused only on neuronal function, recent research has revealed that glia also can exert a substantial influence on experience-dependent brain development. We have found that oligodendrocytes, a type of glial cell that produces myelin in the central nervous system, are sensitive to social experience, and their response to this experience can lead to profound differences in brain function. Considering our work together with that recently described by another group, we discuss social experience-dependent myelination in the prefrontal cortex, especially as it relates to attention deficit hyperactivity disorder and pervasive developmental disorder, two maladies where the function of the prefrontal cortex is disturbed.

View full abstract

Microglia are intrinsic immune cells which release factors, including proinflammatory cytokines, nitric oxide (NO) and neurotrophins, following activation after disturbance in the brain. There is increasing evidence suggesting that pathophysiology of neuropsychiatric disorders such as schizophrenia and depression is related to inflammatory responses mediated by microglia. Microglial activation might not be the primary cause of neuropsychiatric disorders but could be closely related to the pathophysiology of acute stage of neuropsychiatric disorders. We and other researchers have recently shown the inhibitory effects of some antidepressants as well as antipsychotics on the release of inflammatory cytokines and NO from activated microglia, possibly through the modulation of intracellular Ca2+ signaling. Brain- derived neurotrophic factor (BDNF) is a neurotrophin well known for its roles in the activation of microglia as well as in pathophysiology and/or treatment of neuropsychiatric disorders. BDNF also modulates microglial intracellular Ca2+ signaling and precursor BDNF (proBDNF) could play important roles in microglial functions and could also be involved in the pathophysiology of neuropsychiatric disorders.

View full abstract