One of the most remarkable features of human cognition is our ability to derive generalizations from a limited set of examples.
We make such inductive leaps when learning the grammatical structure of a new language, evaluating a restaurant after a single meal, or drawing inferences about the beliefs of others after just a couple of conversations. The need to make inferences about the world around us on the basis of sparse and often ambiguous data is common to all animals, and requires the construction of an internal model of the environment’s governing rules—in essence, a framework of beliefs that can be refined by experience. Little however is known about how an animal’s beliefs about its environment are represented in neural circuits. The overall interest of our lab is to understand how an animal’s beliefs about its environment are represented in, and model-based inference is accomplished by, neural circuits.
Recent work has suggested that the anterior cingulate cortex (ACC) is a likely candidate region for implementing such computations. The main focus of our own work over the past six years has been to explore the role of rodent ACC in encoding an internal model of the environment’s governing rules.
To search for evidence of model encoding in ACC, we designed a behavioral task that requires rats to adjust to abrupt changes in the value of two options, rewarded stochastically (Figure 1A). This task provides not only a read-out of an animal’s transition to exploration, but also enabled us to distinguish activities encoding internal beliefs, sensation, perception or behavioral output. Using extracellular recordings of population activity during this task, we found that abrupt, marked and widespread changes in activity occur in the rat ACC neural network when an animal detects a change in the environment, suddenly abandons his old beliefs and re-explores available options (Figure 1B). These network resets occur at the onset of exploratory bouts and involve an abrupt change in the pattern of activity within ACC: individual neurons increase or decrease their firing rate, yet there is no change in the net firing rate across the entire ensemble. These resets reflect a switch in network activity to a state of instability, which diminishes during the period of exploration (Figure 1C). Thus, ACC ensemble dynamics and an animal’s beliefs track together: periods of confidence in a particular hypothesis associated with a relatively stable behavioral strategy are marked by relative stability in ACC ensemble activity, whereas exploratory periods are marked by volatility in ensemble activity in ACC (Figure 1D). You can find more details about this study here.
Persistence with a behavioral strategy in the face of inadequate beliefs about environmental rules can often be detrimental. Our Karlsson et al. study, as well as a number of studies from other labs that also used behavioral settings in which task rules change suddenly, have provided clues that internal models can be overridden. In such settings, animals respond to rule changes by abruptly initiating exploratory behavior, implying the existence of a mechanism that can release behavioral control from the influence of an internal model that has been deemed inadequate. To search for such a mechanism, we turned to a setting, in which an animal’s modeling attempts can be persistently thwarted by a sophisticated virtual competitor (Fig. 2A). A muscimol-mediated inactivation approach revealed that in this behavioral mode the normal engagement of anterior cingulate cortex (ACC) is suppressed (Fig. 2B).
In a parallel effort that relied on a specially-designed behavioral experiment that probes whether animals can detect and exploit novel opportunities in the environment, we demonstrated that when faced with a competitor that they cannot defeat by counter-prediction, animals switch to a distinct mode of action selection in which they ignore the statistics of the reward associated with different choice patterns and thus make choices independent of their prior experience (Fig. 2C). Moreover, selective enhancement or suppression of Locus Coeruleus input into the ACC, respectively, abolished or restored model-based control of behavior and, with it, sensitivity to environmental feedback (Fig. 2D). Our findings suggest that under conditions of uncertainty about environmental rules, changes in noradrenergic input alter ACC output and prevent erroneous beliefs from guiding decisions, thus enabling behavioral variation. You can find more details about this study here.
Together, our findings point to a key and selective role for ACC in generating, evaluating, and refining the internal model—a set of hypotheses about the rules of the environment—under conditions of uncertainty, and argue that noradrenergic action in ACC controls the extent to which behavioral choices are guided by the internal model or made independently of prior experience and beliefs. In the coming years we will attempt to unravel just how the internal model is represented by ACC ensemble activity, and the exact role of the noradrenergic modulation in model updating, using a combination of tailored behavioral experiments, advanced electrophysiological and optogenetic methods, and innovative analysis techniques.
In parallel with the major effort in the lab to answer these important biological questions, we devote significant resources to the development of new tools and instrumentation with the aim of facilitating experiments that focus on dissecting neural circuit basis of complex behaviors in rodents. These range from molecular tools (in part in collaboration with Loren Looger, Josh Dudman and Adam Hantman) to hardware that simplifies the goal of monitoring and perturbing neural activity during complex behaviors (in part in collaboration with a lab alumnus Mattias Karlsson, now at SpikeGadgets, as well as Janelia’s Instrument Design &Fabrication group and Applied Physics & Instrumentation Group).