Bayesian Methods

Studying individual differences in psychology often involves examining correlations across various measures. However, research involving high-dimensional data—such as in task batteries or neuroscience—often targets latent constructs rather than individual correlations. Furthermore, the number of correlations grows quadratically with increasing dimensionality, potentially leading to overfitting and spurious inference. Therefore, researchers commonly use factor analysis to study individual differences. However, conventional approaches ignore the hierarchical structure of the data and overlook measurement error, leading to attenuated factor loadings. We introduce a Bayesian framework that integrates hierarchical modeling to account for measurement error with factor analysis to infer latent structures. The framework employs novel techniques in the field of Bayesian factor analysis to facilitate model comparison and reduce a priori constraints. The accompanying software enables the creation of generative models at the individual level, supporting a wide range of hypotheses—from descriptive to theory-driven models—and facilitating robust group-level inferences grounded in psychological theory. Through simulations and empirical applications, we demonstrate that our hierarchical factor analysis method flexibly and reliably estimates latent structures in high-dimensional data, offering a valuable tool for individual- differences research in psychology and neuroscience.

This talk presents the rationale and methods for doing distribution-free Bayesian statistical analyses with the Barch and Chechile (2023) DFBA R package. The DFBA package provides tools for conducting Bayesian counterparts to several common frequentist nonparametric methods - including the Mann-Whitney U test, the Wilcoxon Signed-Rank test, the Kendall Tau correlation, and more - as well as functions to assist with experimental planning and distribution-free model selection. We also discuss some new software functions that are in development; these functions provide Bayesian methods for doing a survival analysis in a biomedical context and address the generalization of bivariate statistical association to include multiple regression and stepwise regression. Examples of all the new statistical procedures are discussed, and comparisons are drawn about how these methods improve upon existing frequentist analyses.

Individual differences in behavioral performance can offer key insights into the mechanisms underlying cognitive abilities. Within this framework, the central focus is on examining patterns of correlations across a range of tasks, conditions, and participants. These patterns may reveal underlying factors or clusters, offering a more nuanced characterization of cognitive abilities. Measuring these patterns, factors, and clusters is difficult in real-world settings due to excessive trial noise and measurement error. These difficulties are naturally accounted for by Bayesian hierarchical factor models, which capture individual differences, trial noise, and task variability. However, key challenges remain: First, dimensional regularization is needed to zero out substantially irrelevant factors. Second, a method of principled model comparison is required to evaluate competing latent structures. Third, rotational ambiguity presents a key challenge, as structures may rotate from iteration to iteration in MCMC sampling. We present progress in addressing these challenges and illustrate it through an analysis of individual differences in visual illusions. The visual-illusion case is pertinent because previous research has found small correlations, which remain uncertain in their interpretation. The hierarchical model provides disattenuated measures of correlation, along with uncertainty estimates reflecting variability across individuals, trials, and tasks. We show that although correlations are indeed small, they are well localized, and the overall pattern reveals a single factor, which may be interpreted as a general susceptibility to visual illusions.

Bayesian inference provides a principled framework for modeling cognitive and psychometric data, but scalability remains a challenge. Traditional MCMC methods become computationally impractical when working with very large datasets. In these scenarios, MCMC methods often require extended periods of continuous computing and in many cases result in defective, non-convergent chains. Divide-and-conquer (DC) methods offer a scalable alternative by partitioning data into disjoint subsets, performing computations separately on each, and then recombining the “subposterior” MCMC samples to approximate the full posterior distribution. Crucially, the recombination strategy in use directly affects posterior accuracy and predictive performance. Here, we evaluate the performance of different recombination strategies across full and partitioned datasets in cognitive and psychometric models, including Item Response Theory models. Comparing DC recombination against full-data inference allows us to explore the degree in which different strategies accurately track the target posterior. Using this approach, we systematically compare trade-offs between computing costs and posterior accuracy, highlighting the conditions under which different recombination rules succeed or fail. Our analysis considers factors such as the size of the dataset and the type of model, providing insights into the practical limitations of DC methods for scalable Bayesian inference.