Clinical Models

We propose a one-parameter model of the autism spectrum. This parameter, w_s, scales the self-excitatory weights of neural network units in a model of arbitrary-N-choice decision making and operant conditioning. A larger w_s corresponds to greater symptom severity. The neural net reduces to a diffusion model with thresholds and drift determined by reward rate estimates. Increasing w_s increases the effective time constant of reward integration and maximizes earnings in environments with slower changes in reward contingencies. We lack an analytical solution for the model’s behavior, so we simulated it over a range of choice set cardinalities, from 2 to 100 options, and over ranges of other nuisance parameters. Choice behavior became more exploitative and less exploratory as w_s increased, capturing the preference for routine and repetitive behavior that constitute one of two defining features of autism spectrum disorder. Although the model implements a choice function that approximates a softmax function of reward, it is not the softmax temperature that determines its exploration/exploitation tradeoff — instead it is the time constant of reward rate estimates feeding into the softmax function and governed by w_s that determines it. The reward-maximizing w_s also depends on the environment, decreasing toward the neurotypical end of the spectrum as the reward environment becomes more dynamic. The implication is that neurodiversity along the autism spectrum allows human populations to host individuals with a range of biases about reward-estimation time constants, allowing populations to adapt to environments with a genetically unpredictable range of reward contingency dynamics.

Computational modeling of the Iowa Gambling Task (IGT) has emerged as a powerful tool for decomposing decision-making processes into interpretable parameters that attempt to capture distinct mechanisms. Through a systematic review of 28 studies, we examined how applying computational modeling of the IGT to substance use populations contributes to understanding impaired decision-making. Our analysis traces the theoretical evolution from simple expected value updating to frameworks incorporating key reinforcement learning concepts (e.g., non-linear utility functions capturing subjective value). Although only 25% of studies performed explicit model comparisons, these theoretical additions improved model fit and psychological interpretability. The most robust finding across frameworks was that substance users show marked insensitivity to losses, beyond their heightened reward sensitivity. Studies demonstrated that while non-substance using groups show non-linear growth in loss utility, substance using group’s loss utility remains near zero regardless of magnitude. Evidence for cognitive difficulties (captured by recency parameters) was stronger in models using decay-reinforcement learning rules compared to delta rules, highlighting how theoretical choices in model specification can reveal or obscure underlying mechanisms. These findings demonstrate how evolving cognitive models can reveal mechanistic insights not apparent from behavioral analyses, while emphasizing the importance of theoretically-motivated model selection in understanding decision-making processes in relation to real-world issues like addiction.

The reinforcement learning and decision-making framework has been instrumental in uncovering the neurocognitive processes that underlie human behaviors and psychopathology. However, traditional approaches often fall short in closely simulating real-world behaviors due to their oversimplified nature. To address the limitation, my lab has utilized naturalistic paradigms, including real-time driving and movie-watching tasks. In one study, a real-time driving task was employed to investigate impulsivity. Behavioral performance from the driving task strongly correlated with self-reported impulsivity but not with traditional laboratory measures of impulsivity. Using an inverse reinforcement learning (IRL) algorithm integrated with deep neural networks, we could infer dynamic reward values during the task. Participants with greater impulsivity attributed higher subjective rewards to risky situations. A follow-up fMRI study (n=45) demonstrated that IRL-inferred rewards tracked neural activity in the reward circuit, validating IRL as a tool for modeling RL in real-time. Another study employed a naturalistic video-watching paradigm to investigate alcohol craving. Alcohol users (n=53) verbally reported their drinking reasons, which informed their individual alcohol-related schemas. During fMRI scans, participants watched videos with alcohol cues and rated their craving and self-relatedness. Results revealed that self-relatedness mediated the link between addiction severity and craving. Neural analyses showed that individuals with similar schemas exhibited synchronized activity in a craving-related regions. Conversely, divergent schemas were associated with reduced neural synchrony, highlighting schema-driven differences in craving-related brain activity. These studies exemplify the potential of naturalistic paradigms and advanced computational approaches to simulate real-world situations and characterize individual differences, offering novel insights into addiction.