Understanding brain-computational mechanisms by testing deep neural network models with massively multivariate brain-activity data
Recent advances in neural network modelling have enabled major strides in computer vision and other artificial intelligence applications. Artificial neural networks are inspired by the brain and their computations could be implemented in biological neurons. Although designed with engineering goals, this technology provides the basis for tomorrow’s computational neuroscience, engaging complex cognitive tasks and high-level cortical representations. We are entering an exciting new era, in which we will be able to build neurobiologically faithful feedforward and recurrent computational models of how biological brains perform high-level feats of intelligence.
The objective of the lab is to understand the brain information processing that enables visual perception, object recognition, and scene understanding. Vision is of interest in its own right, but also provides a model for understanding, more generally, how the brain computes and how it might perform probabilistic inference through parallel and recurrent computations.
The lab uses massively multivariate measurements of brain activity along with behavioural data to test models of brain information processing that perform visual tasks. To explain visual processing, the models must meet computational challenges comparable to those biological visual systems face in the real world. The models therefore need to contain rich visual knowledge about the world and have substantial computational power. Building such models requires the methods of machine learning and artificial intelligence. We take a top-down approach to modelling, starting with models that perform the task, but abstract from much of the biological detail. We then attempt to reveal the aspects of human task performance and brain activity that these models fail to explain. This motivates adjustments to the architecture and the design of the units. Architectures and units must be plausibly implementable with biological neurons. Their design is chosen as required by function and inspired by biology, so as to better explain brain and behavioural data. The lab develops neural net models, statistical inference and visualisation techniques, and visual stimuli and tasks, and measures brain activity with fMRI and MEG in humans and with array recordings in nonhuman primates.
Computations in human sensorimotor control
Movement is the only way we have of interacting with the world, whether foraging for food or attracting a waiter's attention. Indeed, all communication, including speech, sign language, gestures and writing, is mediated via the motor system. Taking this viewpoint, the purpose of the human brain is to use sensory signals to determine future actions. The goal of our lab is to understand the computational principles underlying human sensorimotor control. We use theoretical studies, computer simulations and human experiments to examine the basis of skilled motor behavior.
Our focus is on the control of the hand and arm as a model system that demonstrates many of the features which make sensorimotor control hard. The upper limb has a large number of interacting degrees of freedom and it interacts with many different objects under a variety of environmental conditions. Despite this complexity, healthy humans demonstrate a remarkable ability to generate accurate and appropriate motor behavior when interacting with the world. To examine the computations underlying sensorimotor control, we have developed a research program that uses computational techniques from machine learning, control theory and signal processing together with novel experimental techniques that include robotic interfaces and virtual reality systems that allow for precise experimental control over sensory inputs and task variables.
There are five broad areas to our research program: (1) Motor planning and optimal control, (2) Probabilistic models of sensorimotor control, (3) Predictive models for estimation, control & sensory processing, (4) Motor learning of novel dynamics, and (5) The interplay between decision making and sensorimotor control.
Behavioral and cognitive neuroscientific investigations of memory encoding, consolidation and retrieval
How do we form lasting memories of our everyday experiences? In the lab, we want to understand how experiences are initially encoded, undergo further consolidation and are later retrieved. We use behavioral and neural (conventional and high-resolution fMRI, iEEG, MEG) measures to help us learn more about the cognitive and neural operations that contribute to episodic memory.
How are memories formed? We have focused on understanding how the brain and, in particular, the medial temporal lobe (MTL) encodes our experiences. Our main approach has been to examine brain activation in MTL substructures during an experience and to identify patterns of activation that are associated with successful memory formation. We are particularly interested in how we build memories that allow us to later reconstruct the episodic details (the what, when and where) of the past.
Recently, we have also focused on understanding how our perception of event structure (i.e. segmentation) modulates both how those events become organized in memory and the neural processes used to bind information within and across events. Perception, attention, working memory and prediction all interact with encoding processes to determine what will be remembered and how it will be linked with other aspects of our ongoing experience.
Memory consolidation refers to post-encoding brain activity that strengthens the representation of an encoding event. We have focused specifically on looking for reactivation, or replay, or prior encoding events during post-encoding rest periods. While sleep has been linked with memory consolidation and memory integration, we are interested in whether some aspects of our daily experience will be replayed while we are awake! Wouldn’t that be efficient? We are also interested in the role of conscious reactivation (or retrieval) after a period of consolidation and how this can improve subsequent consolidation.
Prevailing theories of episodic memory propose that retrieval is supported by recreating the patterns of activity in the brain that were there during the original experience. It has been proposed that reactivation of cortical regions involved in the original encoding is mediated by hippocampal pattern completion triggered by the memory cue. Recent work in the lab has examined the ways in which encoding processes and retrieval goals may modulate this reactivation.
In her doctoral research, Dr. Gelinas studied cellular mechanisms of learning and memory in the hippocampus. Her postdoctoral fellowship was with Dr. Gyorgy Buzsaki at New York University Langone Medical Center, investigating the effects of epileptic activity on neural networks involved in cognition, as well as advanced neural interface devices for the diagnosis and treatment of epilepsy. In her current research, Dr. Gelinas is focused on understanding how epileptic activity disrupts the proper development and function of neural networks. In vivo neurophysiology with advanced neural interface devices, behavioral memory tasks, responsive stimulation of neural networks, and neurocomputational methods are among the techniques used in her laboratory to investigate neural network dysfunction in epilepsy. The overall goal of her research is to identify novel biomarkers and systems level treatments for epileptic disorders, especially those affecting neonates and children.
Dr. Gelinas is an assistant professor of neurology (in the Institute for Genomic Medicine and the Gertrude H. Sergievsky Center) at Columbia University Medical Center. Dr. Gelinas obtained her medical doctorate and doctorate degrees at the University of Alberta, Canada. She subsequently completed pediatric neurology residency at the University of British Columbia, followed by an epilepsy fellowship at New York University Langone Medical Center. Dr. Gelinas' clinical practice focuses on infantile and childhood epilepsy, with a special interest in epilepsy surgery and intracranial electroencephalography (iEEG).
Dr. Khodagholy's research aims to use unique properties of materials for the purpose of designing and developing novel electronic devices that allow efficient interaction with biological substrates, specifically neural networks and the brain. This process involves design, characterization, and fabrication of high-performance biocompatible electronics to acquire and analyze neural data. The ultimate goal is to translate such advances in electronics, materials and neuroscience into more effective diagnostics and treatments for neuropsychiatric diseases.
Dr. Khodagholy is an assistant professor in the Department of Electrical Engineering, School of Engineering and Applied Science at Columbia University. He received his Master’s degree from the University of Birmingham (UK) in Electronics and Telecommunication Engineering. This was followed by a second Master’s degree in Microelectronics at the Ecole des Mines. He attained his Ph.D. degree in Microelectronics at the Department of Bioelectronics (BEL) of the Ecole des Mines (France). He completed a postdoctoral fellowship in systems neuroscience at New York University, Langone Medical Center.