David S. Johnson

David S. Johnson

Human-AI collaboration is often proposed to improve high-stakes decision-making, yet the influence of increased stakes and imperfect AI on decision-making strategies is not fully understood. Studying such behavior in realistic settings is challenging, as application-grounded evaluations are costly, rely on experts, or lack meaningful consequences for decision errors. To address this, we introduce Blockies, a parametric dataset generator for visual diagnostic tasks, and conduct an empirical study examining how perceived stakes influence reliance calibration and behavior. Results show that raised stakes lead to longer deliberation, but less calibrated reliance, with participants increasingly deferring to incorrect AI advice as decision time increased. These findings highlight that increased effort under higher stakes does not necessarily improve reliance calibration and show the importance of accounting for stakes when evaluating human-AI decision-making.

David S. Johnson, Olya Hakobyan, Hanna Drimalla

Machine learning is frequently used in affective computing, but presents challenges due the opacity of state-of-the-art machine learning methods. Because of the impact affective machine learning systems may have on an individual's life, it is important that models be made transparent to detect and mitigate biased decision making. In this regard, affective machine learning could benefit from the recent advancements in explainable artificial intelligence (XAI) research. We perform a structured literature review to examine the use of interpretability in the context of affective machine learning. We focus on studies using audio, visual, or audiovisual data for model training and identified 29 research articles. Our findings show an emergence of the use of interpretability methods in the last five years. However, their use is currently limited regarding the range of methods used, the depth of evaluations, and the consideration of use-cases. We outline the main gaps in the research and provide recommendations for researchers that aim to implement interpretable methods for affective machine learning.

David S. Johnson, Wolfgang Lorenz, Michael Taenzer et al.

Research on sound event detection (SED) in environmental settings has seen increased attention in recent years. The large amounts of (private) domestic or urban audio data needed raise significant logistical and privacy concerns. The inherently distributed nature of these tasks, make federated learning (FL) a promising approach to take advantage of largescale data while mitigating privacy issues. While FL has also seen increased attention recently, to the best of our knowledge there is no research towards FL for SED. To address this gap and foster further research in this field, we create and publish novel FL datasets for SED in domestic and urban environments. Furthermore, we provide baseline results on the datasets in a FL context for three deep neural network architectures. The results indicate that FL is a promising approach for SED, but faces challenges with divergent data distributions inherent to distributed client edge devices.