last updated 6/29/2014
CCNLab/publications offers an up-to-date summary of recent lab publications. The online textbook provides an overview of our approach in the first chapter and extensive mentions of prior research; however, it is written at an undergraduate level.
See http://www.archive.org/details/ucb_icbs_2009_11_13_Randy_OReilly for a video of a talk describing recent lab research.
- 1 Overall Approach
- 2 Narrative and Specific Topics
- 3 Older Research Summaries
- 3.1 Role of the Hippocampus and Neocortex in Learning and Memory
- 3.2 Role of the Hippocampus and Neocortex in Recognition Memory
- 3.3 Role of the Hippocampus and Neocortex in Animal Learning Phenomena
- 3.4 Nature of the Representations and Learning in the PFC
- 3.5 Nature of Cortical Learning and Processing
- 3.6 Summary Statement of Research Interests
This figure summarizes the overall strategy and mission of the CCNLab. We develop and refine a core set of computational mechanisms that capture the underlying functional properties of the major brain areas (neocortex, hippocampus, basal ganglia, cerebellum, brainstem neuromodulatory areas, etc), and use these mechanisms to understand a wide range of different cognitive and behavioral phenomena. These small set of core mechanisms give us great leverage in terms of how wide a range of distinct phenomena we can explain. We delve deeply into specific domains, to provide a stringent and informative test of these mechanisms, and apply what we learn back to refining and further developing our core set of computational mechanisms. Critically, we eschew the common practice of introducing specific mechanisms that explain specific phenomena in a given domain, in a transparent and simple(istic) manner, in favor of leveraging our core mechanisms, which depend critically on learning as the major source of cognitive function. If we cannot explain how the brain learns to perform a specific cognitive function using domain general, biologically-based learning mechanisms, then we have failed.
In terms of levels of analysis, we seek a synergy between higher-level computational and mathematical principles (which ensure that our models work -- they do powerful, efficient forms of learning and information processing), and the underlying biology of the brain areas in question (which ensures that our models are capturing important aspects of the true function of the brain -- not just wishful thinking or purely artificial intelligence). We have repeatedly found such synergies, where important computational principles that enable powerful forms of learning and processing can be directly tied to specific biological mechanisms. Fundamentally, we believe that the brain is a powerful learning and information processing mechanism, and with the right computational lens, we can understand what the detailed biology is doing to achieve these functions.
Our primary goal is scientific: we seek to understand our own brains. But we also believe that in so doing, particularly with our computationally-focused approach, we will be capable of replicating the incredible power of our brains in artificial systems. Thus, we are very interested in developing practical applications of our systems, where applicable, and have found that a deep exploration of model performance on specific tasks of practical interest can be very informative for feeding back into the development of our core computational mechanisms. Ultimately, we believe that our work can contribute to the development of a complete whole-brain model that exhibits human-scale (and beyond!) cognitive abilities. Furthermore, we believe that our approach is the most likely to get there first.
Narrative and Specific Topics
How does the brain think? This is one of the most challenging unsolved questions in science, and it is the focus of the CCN Lab's research in the field of cognitive neuroscience. This question is challenging because there are billions of neurons in the brain, and they interact in complex ways to produce behavior, which is itself complex and varied. A wide range of different research methods have been developed in response to the challenging nature of this problem, with the hope of obtaining converging evidence from multiple methodologies. Thus, to make progress in cognitive neuroscience, a researcher must master a range of methods and content domains spanning neuroscience and cognition, and must engage in highly collaborative research with experts in different methods and domains. These are important characteristics of research in the lab.
The specific contribution of the CCN Lab's research to cognitive neuroscience is in developing theoretical models that can provide a principled understanding of how networks of interacting neurons can give rise to cognitive phenomena. The goal is to "span the gap" between brain and behavior by capturing the essential properties of neural networks in computer models, and testing these models on their ability to reproduce a wide range of cognitive phenomena, including perception, learning and memory, language processing, and higher-level cognition. Therefore, in addition to cognition and neuroscience, people in the lab have to be proficient in computer simulation and computational/mathematical analyses of complex systems. Lab members also test key predictions of models using behavioral or other methods, and in collaboration with other researchers.
The kinds of theoretical models developed by the lab play an important role in cognitive neuroscience, because it is very difficult to understand the complexity of neural systems and their cognitive capabilities based on more simplistic verbal theories. For example, theorizing in traditional cognitive psychology has been typically based on "box and arrow" models containing a set of labels for processing operations in the boxes (e.g., "object recognition," "short term memory buffer," and "lexicon") connected by arrows that indicate some kind of information flow. Other types of theories have relied on the computer metaphor, having a "CPU" and various kinds of short and longer-term storage buffers. In contrast, neural networks like those found in the brain operate in a massively parallel fashion where many neurons work together, each making a small contribution. Furthermore, memory and processing are embedded inextricably within each of these neurons, instead of being segregated as in a standard computer. Information in these networks is not simply passed around, but is instead embedded in distributed patterns of neural activation that directly influence processing in other parts of the brain. New kinds of thinking are required to understand how these neural systems behave, and developing these new kinds of thinking is the primary focus of our research.
The research contributions can be categorized in the following ways:
- Developing a set of basic principles for the models by attempting to simultaneously satisfy constraints from biological, cognitive, and computational levels of analysis. The result is the Leabra framework, which combines standard integrate-and-fire style point neurons with powerful error-driven and associative/self-organizing learning mechanisms, and a highly optimized k-winners-take-all inhibitory dynamic that produces sparse distributed representations. We have recently been developing increasingly biologically detailed models for how the crucial error-driven learning mechanism functions in the brain, building on biophysically detailed models of spike-timing dependent plasticity (STDP). A paper on this work is in preparation.
- Understanding how different brain areas have specialized variants of these basic core mechanisms that enable specialized forms of learning and memory. Early work focused on the hippocampus, showing that very sparse representations (high levels of inhibitory competition) in the CA3 and especially DG layers allow this area to rapidly learn new information while minimizing interference. More recent work has focused on the specializations of the prefrontal cortex (PFC) and basal ganglia (BG) that support robust active maintenance, and reinforcement-based dopamine-driven learning.
- Integrating sets of specialized brain-area models into a coherent overall cognitive architecture that can perform complex real-world tasks. The specific focus of this work has been on understanding how learning can occur in a more naturalistic "real-world" environment, using a virtual robotic agent ("emer", who lives in the Emergent neural network simulation software), in a virtual 3D environment with naturalistic physics. A critical enabling functionality for this work is the ability to recognize objects from visual sensors, and also to manipulate these objects and the environment more generally. In short, this work has led us to appreciate the importance of embodiment, and how cognitive function emerges out of more basic sensory-motor learning capabilities. Emer can currently recognize over 100 object categories, and generalize with high levels of accuracy (> 90%) to novel exemplars. Motor control over the visual saccade system is achieved through a simulated cerebellum (rounding out the full complement of major brain areas), and reaching and grasping work is currently under way. Future work includes incorporation of the auditory stream of sensory processing, enabling initial explorations of the foundations of language learning, elaboration of the spatial navigation abilities in conjunction with a parietal cortex and the hippocampus, and greater incorporation of the PFC/BG system to support cognitive control, planning, problem solving, etc.
- Developing a synthesis of the Leabra framework and the ACT-R framework, which provides a more abstract, higher-level description of a cognitive architecture that overlaps considerably with the Leabra architecture. This "SAL" (synthesis of ACT-R and Leabra) model seeks to achieve better capabilities than either existing architecture alone, by synthesizing the best elements of each.
Older Research Summaries
Role of the Hippocampus and Neocortex in Learning and Memory
A major thrust of my research is on understanding the complementary roles of the hippocampus and the neocortex in learing and memory. In collaboration with Jay McClelland and Jerry Rudy, I have elaborated a set of basic principles of how the hippocampus differs from the neocortex, and I have captured these principles in computational models of these systems. The current research focuses on applying these ideas in two domains: human recognition memory and animal learning phenomena.
For more information on the basic principles of hippocampal and cortical function, see:
- O'Reilly, R. C. & Rudy, J. W. (2000). Computational Principles of Learning in the Neocortex and Hippocampus, Hippocampus, 10, 389-397
- O'Reilly, R. C. (in press). The Division of Labor Between the Neocortex and Hippocampus Connectionist Modeling in Cognitive (Neuro-)Science, George Houghton, Ed., Psychology Press
- McClelland, J. L., McNaughton, B. L. & O'Reilly, R. C. (1994/1995). Why There Are Complementary Learning Systems in the Hippocampus and Neocortex: Insights from the Successes and Failures Of Connectionist Models of Learning and Memory Technical Report PDP.CNS.94.1, March 1994, Psychological Review, 102, (1995), 419-457.
- O'Reilly, R. C. & McClelland, J. L. (1994). Hippocampal Conjunctive Encoding, Storage, And Recall: Avoiding A Tradeoff Technical Report PDP.CNS.94.4, June 1994, Hippocampus, 4, 661-682.
Role of the Hippocampus and Neocortex in Recognition Memory
In collaboration with Kenneth Norman, we have developed a fully functional model of all the major components of the hippocampus, and have applied this initially to understanding its role in recognition memory. Recognition memory is typically tested by presenting lists of words to subjects, and then asking them later if they remember seeing a given set of test words during the initial study session. The hippocampus can rapidly encode many aspects of the experience of studying the word and bind them together in one conjunctive representation. This can then be used to explicitly remember or recollect the study experience during testing. This recollective contribution of the hippocampus has many distinct properties compared to an alternative familiarity based response, where the subject has some general feeling of familiarity, but no distinct recollection of studying it. This familiarity response is likely due to the effects of cortical learning. We are trying to understand how the unique biological properties of the hippocampus and cortex give rise to their different functional properties as measured in behavioral tests of recognition memory.
Dave Huber and I are trying to use the same principles that underlie the cortical contribution to familiarity to understand priming effects taking place in lower-level cortical areas.
For more information on this project, see:
- Norman, K. A. & O'Reilly, R. C. (in press). Modeling Hippocampal and Neocortical Contributions to Recognition Memory: A Complementary Learning Systems Approach. ICS Technical Report 01-02, August 2001. Psychological Review, in press.
- O'Reilly, R. C. & Norman, K. A. (2002). Hippocampal and Neocortical Contributions to Memory: Advances in the Complementary Learning Systems Framework. Trends in Cognitive Sciences, 6, 505-510.
Role of the Hippocampus and Neocortex in Animal Learning Phenomena
In collaboration with Jerry Rudy, we are applying the hippocampal model and a model of cortical learning to understand the learning of conjunctive representations in a wide range of animal learning paradigms. Specifically, we have developed an understanding of why the conjunctive hippocampal representations appear to be necessary for certain conditioning tasks that would seem to require the use of conjunctive representations, but not others. The model also suggests that a number of other tasks would provide better indicators of hippocampal function, and we are presently exploring these both in the lab with rats and in the model.
For more information on this project, see:
- O'Reilly, R. C. & Rudy, J. W. (1999/2001). Conjunctive Representations in Learning and Memory: Principles of Cortical and Hippocampal Function. ICS Technical Report 99-01, January 1999 (Revised June 2000). Psychological Review, (in press)
- Rudy, J. W. & O'Reilly, R. C. (2001). Conjunctive Representations, the Hippocampus and Contextual Fear Conditioning. Cognitive, Affective, and Behavioral Neuroscience, 1, 66-82.
Nature of the Representations and Learning in the PFC
In collaboration with Jonathan Cohen and Todd Braver, we are developing models of the prefrontal cortex (PFC) that explore how biological properties of the PFC can contribute to its unique role in working memory and controlled processing (aka, "executive" function). These models incorporate a role for the neuromodulator dopamine in controlling the updating and maintenance of information in the PFC, which is one important component in enabling a form of activation-based processing that is much more flexible and dynamic than the kinds of weight-based processing more typical of the posterior cortex. We are exploring these issues in the context of a categorization task that is analogous to the Wisconsin Card Sorting Test (a commonly-used test of frontal function), which requires that the categorization rules be rapidly switched. Frontal damage results in slowing (perserverations) in switching, which can be explained by assuming that less dynamic weight-based processing is being used instead of the activation-based processing supported by the PFC in normals.
The flexibility of the PFC, plus the need for robust maintenance, also places significant requirements on the nature of the representations in the PFC. Initial progress towards understanding these representations has been made in the context of the categorization task in work with Cohen and Braver. More recently, work with Miyake, Mozer, and Munakata has begun to explore the specific idea that PFC representations should be more discrete in nature, which affords an important level of robustness. Consider the example of representing colors. A continuous representation would encode something like the wavelength of the light. A discrete representation would be more like the familiar color words, "red", "blue", "green", etc. By maintaining a discrete representation, the effects of noise can be minimized because discrete attractor structures can constantly pull the representation back towards a discrete attractor. In contrast, such attractors are impossible for a continuous space such as wavelength, so that continuous representations would be subject to a random walk under the effects of noise (and in the absence of external inputs, as is typically the case where active maintenance is required). We are exploring this idea, and the consequent tradeoffs between continuous and discrete representations, in simulations and in behavioral experiments with adults, infants, and frontal patient populations.
For more information on this project, see:
- Rougier, N. P. & O'Reilly, R.C. (2002). Learning Representations in a Gated Prefrontal Cortex Model of Dynamic Task Switching. Cognitive Science, 26, 503-520.
- O'Reilly, R. C., Noelle, D. C., Braver, T. S., & Cohen, J. D. (2002). Prefrontal Cortex and Dynamic Categorization Tasks: Representational Organization and Neuromodulatory Control. Cerebral Cortex, 12, 246-257
- Frank, M. J., Loughry, B. & O'Reilly, R. C. (2001). Interactions Between Frontal Cortex and Basal Ganglia in Working Memory: A Computational Model. Cognitive, Affective, and Behavioral Neuroscience, 1 137-160.
- O'Reilly, R. C., Braver, T. S., & Cohen, J. D. (1999). A Biologically-Based Computational Model of Working Memory. Models of Working Memory: Mechanisms of Active Maintenance and Executive Control., Miyake, A. & Shah, P. (Eds), New York: Cambridge University Press.
Nature of Cortical Learning and Processing
My Ph.D. thesis was on a model of cortical learning called Leabra. I have now developed a much simpler and more biologically based version of the central ideas developed in the thesis. Because this algorithm combines many of most important existing computational ideas for learning and processing in neural networks, it provides a useful pedagogical tool for the textbook that Yuko Munakata and I have writen together. This textbook contains simulations of a wide range of cognitive phenomena using the Leabra algorithm. I am also working on several journal articles on the models developed for the textbook. In addition, I use it for most of my other research simulations. While relatively stable at this point, I continue to be interested in developing and extending the ideas embodied in this algorithm, and the large space of cognitive models that I have developed using it.
I now have funding (from the ONR) to explore the ability of this framework to develop useful, systematic representations of structured knowledge domains, starting with simple relational information (e.g., "the triangle is above the square"). This project will be critical for showing that standard distributed representations can handle tasks which many have assumed they could not (and therefore these people have resorted to using explicit, structured, symbolic representations which do not facilitate powerful learning, and are biologically implausible).
For more information on this project, see:
- O'Reilly, R. C. & Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain. MIT Press, 2000.
- O'Reilly, R. C. & Munakata, Y. (2002). Psychological Function in Computational Models of Neural Networks. Handbook of Psychology, Vol 3, Biological Psychology, Michela Gallagher & Randy Nelson, Eds. New York: Wiley
- O'Reilly, R. C. (2001). Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning. Neural Computation, (in press).
- O'Reilly, R. C. (1998). Six principles for biologically-based computational models of cortical cognition. Trends in Cognitive Sciences, 2, 455-462.
- O'Reilly, R. C. (1996). Biologically Plausible Error-driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm. Neural Computation, 8, 895-938.
Summary Statement of Research Interests
Much of cognition occurs principally in the neocortex and related cortical areas such as the hippocampus. One of the most tantalizing properties of the neocortex is the high degree of structural homogeneity --- the same basic neuron types and patterns of interconnectivity are shared across brain areas that subserve vastly different cognitive functions, from perception to language to planning. This homogeneity holds out the promise that a relatively small and consistent set of mechanistic principles could be used to understand the neural basis of a very wide range of cognitive phenomena. Towards this end, I have been working on developing a set of principles that have a clear biological basis, provide satisfying models of cognitive phenomena, and take advantage of powerful computational principles.
My new textbook, Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain (O'Reilly & Munakata, 2000), represents the culmination of this effort to date. In this book, we provide a coherent framework for modeling in cognitive neuroscience (called Leabra) that incorporates a set of six core principles that have been developed separately by a variety of different researchers over the years (as summarized in O'Reilly, 1998). This framework, which represents the evolution of my Ph.D. thesis work, is developed in the text by starting with the basic biological properties of cortical neurons and networks of neurons, and then introducing computational learning mechanisms that can organize these cortical networks to perform useful cognitive functions in response to experience. After providing an overview of the larger-scale ``computational architecture of the brain, we cover core aspects of perception, attention, memory, language, and higher-level cognition through a large number of research-grade simulations, some of which are adaptations of existing models in the literature, and some of which are original models. I am currently in the process of writing up a number of the original models for publication in peer reviewed journals (with several submitted already), and I plan to continue to develop both the basic principles and these models.
I have received funding from the Office of Naval Research to pursue one aspect of further development of these principles, centered around the ability of neural network models to encode complex propositional (a.k.a., structured, relational) information. For example, in encoding ``John loves Mary, one must represent a relationship between the neural encodings of ``John and ``Mary, and distinguish this from many other possible relationships (e.g., ``Mary loves John, which may not necessarily be true!). Furthermore, one should be able to draw many inferences from this fact, and one should also be able to generalize one's knowledge of ``love relationships to novel instances (e.g., ``Raja loves Mujibar). Achieving these objectives in neural networks has long been regarded as a very difficult problem, but initial progress (some of which is in a paper resubmitted to Neural Computation) suggests that the unique combination of principles in the Leabra framework may allow progress where previous attempts have failed. Finally, one of my most important original contributions in developing Leabra was in deriving a form of error-driven learning that is biologically plausible yet computationally very powerful (O'Reilly, 1996) --- I will continue to extend the biological detail of this mechanism in future work.
Cortical and Hippocampal Contributions to Learning & Memory
It has been known for several decades that the hippocampus plays a special role in the acquisition of new memories, but how best to characterize this role has been a matter of continued debate. In work done in graduate school, I helped to develop a set of ideas about the contribution of the hippocampus based on basic principles of neural network function (O'Reilly & McClelland, 1994; McClelland, McNaughton, & O'Reilly, 1995). I have continued to develop these ideas, and have just completed a major paper that addresses a large number of findings regarding the hippocampal contribution to learning and memory (O'Reilly & Rudy, in press). This paper describes for the first time a fully implemented model of simultaneous cortical and hippocampal learning, and shows how this model accounts for the learning performance of both intact and hippocampally-lesioned animals. The model embodies three principled differences between cortical and hippocampal learning (conjunctive bias, learning rate, and sensitivity to task demands) which provide a basis for predicting when hippocampal damage will impair learning, and when it will not. These predictions are supported by current data, and Rudy & I have received funding from the National Institutes of Health to test a number of further predictions of the model on intact and hippocampally-lesioned rats. This work is under way, and should generate a steady flow of publications.
The other major direction of research on this topic, conducted with postdocs Ken Norman and David Huber, and collaborators Andrew Mayes and Tim Curran, involves applying the model to human recognition memory data, and testing its predictions using intact and hippocampally-lesioned patients, and electrical measurements of brain activity. Recognition memory assesses subject's memory for a list of items presented earlier, distinguishing ``studied words from novel ``lure words. Consistent with recent dual-process conceptions of this phenomenon, our model suggests that the cortex and the hippocampus each make distinct contributions to recognition memory (O'Reilly, Norman, & McClelland, 1998), which are testing empirically. For example, we predict that the cortex will exhibit a graded recognition response as a function of the similarity between lures and studied items, whereas the hippocampus will exhibit a more binary response that is informative even with similar lures --- this appears to be the case (Holdstock et al, in press). We are also exploring the interactions between cortical and hippocampal components in cases where they act in opposition, and testing for the widely assumed statistical independence of these systems. In short, we are trying to develop a fully-functional biologically-based model of recognition memory. In addition, we are applying these same principles towards understanding widely-studied priming phenomena --- where recent experience ``primes the pump for subsequent processing.
Prefrontal Cortex Involvement in Higher-Level Cognition
Higher-level cognition includes such important and complex phenomena as planning, problem solving, abstract and formal reasoning, and complex decision making, and represents the frontier of computational cognitive neuroscience modeling. These phenomena are known to depend critically on the prefrontal cortex, although they also involve many other cortical areas (which is why they are so difficult to understand). To address this challenge, my collaborators and I have leveraged a relatively simple and well-established function of prefrontal cortex, called working memory (O'Reilly, Braver, & Cohen, 1999; O'Reilly, Mozer, Munakata, & Miyake, 1999; Cohen & O'Reilly, 1996). Working memory involves actively maintaining information, and manipulating this information for processing purposes. A good example is mental arithmetic --- one must juggle digits, updating sums and maintaining partial products to achieve the correct results. We propose that: (a) the prefrontal cortex is uniquely specialized to perform this working memory function (and we have developed models capturing the precise nature of this specialization); and (b) this basic working memory function underlies all of the more advanced functions of the prefrontal cortex.
My students and I have recently developed a model of the neural interactions between the frontal cortex and the basal ganglia, which are brain areas that play an important role in both motor and cognitive function (and are damaged in Parkinson's disease). We propose that the specialized circuits connecting these brain areas enable the frontal cortex to both rapidly update and robustly maintain information over time, as is required for working memory. The model includes much of the known neurobiology of these systems, and achieves good performance on a cognitive working memory task. Furthermore, it provides a consistent mechanism for both motor and cognitive functions, which is important because the motor aspects are much better understood. We are currently extending the model with powerful learning mechanisms so that the working memory system can adapt to different task demands.
We have also simulated other frontal functions that are not so obviously dependent on working memory. For example, we have developed a novel account of a version of a widely used task for assessing frontal function, the Wisconsin card sorting task, and have simulated effects of lesions in different frontal areas in monkeys performing this task (O'Reilly, Noelle, Braver, and Cohen, submitted). My students and I are currently developing a model of the ``Tower of Hanoi problem solving task. This work is funded by the Office of Naval Research, and the NSF.
In short, I am investigating a range of different cognitive phenomena, but all of my work builds out of a few central principles (e.g., all of my models use the Leabra simulation framework), and therefore remains cohesive. Seeing all of these different pieces fit together is what makes these projects so much fun!