Christopher Edwards

Viktor Kewenig, Andrew Lampinen, Samuel A. Nastase et al.

The potential of multimodal generative artificial intelligence (mAI) to replicate human grounded language understanding, including the pragmatic, context-rich aspects of communication, remains to be clarified. Humans are known to use salient multimodal features, such as visual cues, to facilitate the processing of upcoming words. Correspondingly, multimodal computational models can integrate visual and linguistic data using a visual attention mechanism to assign next-word probabilities. To test whether these processes align, we tasked both human participants (N = 200) as well as several state-of-the-art computational models with evaluating the predictability of forthcoming words after viewing short audio-only or audio-visual clips with speech. During the task, the model's attention weights were recorded and human attention was indexed via eye tracking. Results show that predictability estimates from humans aligned more closely with scores generated from multimodal models vs. their unimodal counterparts. Furthermore, including an attention mechanism doubled alignment with human judgments when visual and linguistic context facilitated predictions. In these cases, the model's attention patches and human eye tracking significantly overlapped. Our results indicate that improved modeling of naturalistic language processing in mAI does not merely depend on training diet but can be driven by multimodality in combination with attention-based architectures. Humans and computational models alike can leverage the predictive constraints of multimodal information by attending to relevant features in the input.

Christopher Edwards, Ralph C Smith

In the event of a nuclear accident, or the detonation of a radiological dispersal device, quickly locating the source of the accident or blast is important for emergency response and environmental decontamination. At a specified time after a simulated instantaneous release of an aerosolized radioactive contaminant, measurements are recorded downwind from an array of radiation sensors. Neural networks are employed to infer the source release parameters in an accurate and rapid manner using sensor and mean wind speed data. We consider two neural network constructions that quantify the uncertainty of the predicted values; a categorical classification neural network and a Bayesian neural network. With the categorical classification neural network, we partition the spatial domain and treat each partition as a separate class for which we estimate the probability that it contains the true source location. In a Bayesian neural network, the weights and biases have a distribution rather than a single optimal value. With each evaluation, these distributions are sampled, yielding a different prediction with each evaluation. The trained Bayesian neural network is thus evaluated to construct posterior densities for the release parameters. Results are compared to Markov chain Monte Carlo (MCMC) results found using the Delayed Rejection Adaptive Metropolis Algorithm. The Bayesian neural network approach is generally much cheaper computationally than the MCMC approach as it relies on the computational cost of the neural network evaluation to generate posterior densities as opposed to the MCMC approach which depends on the computational expense of the transport and radiation detection models.

Thang Nguyen, Christopher Edwards, Vahid Azimi et al.

In this paper, the problem of output feedback sliding mode control of linear sampled-data multi-input multi-output systems is considered. Existing sliding mode control schemes can attenuate the influence of an external disturbance by driving system states onto a sliding surface. However, they can exhibit high gains during transients, which can be $O(1/T)$ where $T$ is the sampling time period. To address this problem, a new strategy, which employs disturbance approximation, is proposed so that the control effort will be $O(1)$. The new method avoids deadbeat phenomena and hence, it will be less sensitive to noise. Theoretical analysis is provided to show the convergence and robustness of the proposed method. Simulations were conducted to show the efficiency of the proposed approach.