Research areas

Neuro-X studies AI that can help us understand and predict human cognition, emotion, behavior, and their changes through brain activity. Our goal is to apply patterns learned from diverse brain data to new people and tasks. We develop a range of foundation models suited to different brain signals and research goals.

Brain activity varies across space and time, and fMRI and EEG record different aspects of it. Models designed around these differences predict cognitive states and mental-health measures; we also analyze what the patterns used for prediction reveal about relationships between brain and behavior. Choices about signal summaries, frequency bands, and electrode configurations serve this goal. Evaluation with new people and recording settings tests where the models can be used.

Learning spatial and temporal patterns in fMRI

SwiFT learns spatial and temporal patterns from fMRI without first reducing it to averages over predefined brain regions. Its successor, SwiFUN, predicts task-evoked brain activity from resting-state fMRI.

In adult UK Biobank and child ABCD data, SwiFUN achieved higher overall similarity between predicted and measured maps than the connectivity-based comparison model, ConnTask. ConnTask performed better at identifying individuals. Predicting shared response patterns and preserving individual specificity therefore require separate evaluation.

MBBN learns from separate frequency bands in fMRI to analyze brain-activity patterns relevant to cognitive and mental-health prediction. It examines how distinguishing different timescales changes the information used for prediction. NeuroMamba uses state-space modeling and removal of non-brain background tokens to reduce the computational cost of fMRI learning. The public study evaluates sex classification; extending it to cognition and mental health remains a goal for further testing.

Learning from EEG across recording settings

DIVER-0 addresses differences in EEG channel configurations. It is designed so that outputs follow changes in channel order, and was evaluated on emotion recognition and motor-imagery classification. DIVER-1 extends this work to intracranial EEG for cognitive tasks and seizure detection.

Building on these models, we study how brain-signal analysis can adapt to new recording settings and limited training data. Combining information from different kinds of brain recordings is another direction within the program.

Why do people differ in how they think, behave, and develop? We seek to understand differences in development and mental health through connections among genes, brain, behavior, and environment. We analyze genetic variation, brain structure and activity, cognition and behavior, and living environments together to study relationships across these levels of observation.

Large developmental datasets and computational models make these relationships open to investigation. We ask which genetic and brain features accompany particular behavioral and environmental measures, what family history captures, and whether models combining different data can help predict development and mental health in new populations. Each study examines particular connections within its own scope, contributing to an account of human development across levels.

Studying genes, brain, behavior, and environment together

Polygenic scores (PGS) combine small effects across many genetic variants to summarize genetic differences associated with a particular trait. Our study of childhood genes, brain, behavior, and environment analyzed PGS alongside several types of MRI, cognitive and mental-health measures, and environmental measures to identify patterns of association.

Environmental measures included family conflict, school engagement, neighborhood safety, and area-level income and poverty indicators. Analyses including brain function also identified patterns related to socioeconomic context. Environment was part of the relationships analyzed across levels, beyond a separate background description.

This approach lets us compare how different brain features relate to genetic, behavioral, and environmental measures. Connecting findings across levels offers clues to understanding development. Patterns of association, however, do not establish a causal path from genes through the brain to behavior.

Our depression family-history study examined what family history and a child’s PGS each contribute to understanding mental health. Depression-related PGS accounted for only part of the association between family history and offspring mental health. The findings support considering family history alongside current genetic measures.

This makes it difficult to treat family history simply as a proxy for genetic liability. Nor can everything left unexplained by PGS be assigned to the environment. Information missing from current genetic measures and environments shared within families still need to be distinguished.

Genetically informed brain-imaging prediction

In our white-matter imaging study, genetic information guided model training to identify depression-related features in brain images. We then evaluated prediction of depression and suicidality measures in separate youth cohorts. The study explores how genetic information and brain imaging can be combined.

Here, genetic information guides pretraining of a brain-imaging model. Testing whether this combination remains useful in new populations is one way to assess progress in studying individual differences.

These studies aim to understand and predict individual differences in development and mental health. The current models do not replace individual diagnosis.

What kinds of computation can learn the complex relationships within brain signals? We combine quantum circuits with classical models to explore new ways of representing and learning from those signals. Applying quantum operations to relationships across time and to learned brain representations lets us ask what different computational structures can contribute to learning.

We test these possibilities by varying circuit size, connectivity, and combinations with classical models. Alongside predictive performance, we examine the number of trainable parameters, constraints on scaling circuits, and training stability under noise. Simulations with brain data and real-hardware experiments on separate tasks test computational methods under different conditions.

Brain time-series analysis

The Quantum Time-series Transformer represents relationships across time using quantum operations and applies them to fMRI analysis. Q-DIVER combines a pretrained EEG model with quantum circuits and searches for circuit designs suited to motor-imagery classification. Both studies evaluated model design and performance in simulation.

Q-DIVER connects quantum circuits to the readout of an already learned EEG representation. It turns the question of which part of a model to assign to quantum computation into a concrete design problem.

Circuit training and hardware

Our multi-chip ensemble study proposes dividing computation among smaller circuits and combining their outputs. It evaluates training stability under simulated noise. Comparing compression into a single circuit with distributing inputs across smaller circuits lets us examine the relationship between representing information and being able to learn from it.

A separate IBM quantum device study runs a handwritten-digit classifier on real quantum hardware.

We evaluate quantum circuits against conventional neural networks. A quantum advantage on brain data has not yet been established.

Multiple emotions can coexist in what we feel and remember, and those emotions change over time. We study how these patterns and changes in experience relate to brain activity. We examine moments of mixed positive and negative feelings during awe and emotional changes during personal recall, while also exploring new media that respond to emotion.

Virtual reality (VR) and experience reports let us study the settings and forms of complex emotions, while EEG records brain activity during those experiences. We analyze how the patterns and changes described in reports relate to neural signals. In a separate recall study, we test how emotional changes estimated from EEG during recall can guide music and video generation.

Awe and mixed feelings

Our study of awe measured emotional changes and EEG while participants watched VR videos. Ambivalent feelings, in which positive and negative affect coexist, were related to the intensity of awe. EEG analyses also identified corresponding patterns. The findings suggest that awe cannot be adequately described as a purely positive emotion.

Allowing positive and negative feelings to be reported together makes experiences visible that a single pleasant–unpleasant scale could miss. The study gives a concrete form to a methodological question: how do the dimensions and scales used to report feelings shape what we can observe?

Emotion during recall and content generation

Revisiting Your Memory is an exploratory study that estimates emotional changes from EEG during recall and uses them to guide music and video generation. The memory content comes from participants’ own writing, drawings, and music. EEG-derived emotional changes help control the generation process; the system does not reconstruct remembered scenes from brain signals alone.

This study explores a meeting point between measuring emotion and making media that respond to it. Participant-provided memory materials and changes estimated from brain signals serve distinct roles in the process.

Related work: methods for visual and musical generation

We also develop methods for controlling visual texture and form, and musical style. The two studies below concern computational methods and do not analyze neural signals or emotional responses.

AesFA separates image features into frequency components to transfer style while accounting for texture and overall structure. Our music style-transfer study represents sound as a time–frequency image and adapts an image diffusion model to change musical style without additional training.