Nelson Hidalgo Julia, Robert Lewis, Craig Ferguson, Joshua Angulo Lopez, Hahrin Jung, Rosalind Picard, Simon Goldberg, Raquel Tatar, Wendy S.-Y. Lau, Caroline Swords, Christine D. Wilson-Mendenhall, Gabriela Valdivia, Molly Schaefer, and Richard Davidson. 2026. Designing an Affective Mobile Probe to Measure Smile Dynamics in Depression. In Proceedings of the 2026 CHI Conference on Human Factors in

Abstract

Depression is a complex disorder for which there is growing interest in identifying objective behavioral markers that measure precise symptoms, such as anhedonia and blunted emotional reactivity. This study explores the feasibility of using smile and smirk expression dynamics, captured through our novel stimulus-based mobile affective probe, as candidate digital biomarkers of depression severity within a large-scale mobile health intervention trial, BeWell. Data from 684 BeWell participants (2,702 observations) are analyzed longitudinally for 16 weeks, comparing their PHQ-8 survey scores with their facial responses to short videos intended to elicit smiles. Mixed-effects models reveal that higher maximum Duchenne smile intensity in reaction to liked stimuli is associated with lower depression scores over time at both within- and between-person levels. We additionally share insights from our tool, including ease of use, perceptions of the stimulus, and technical challenges, which offer considerations for the future development of stimulus-based affect probes in real-world settings.