Real-Time Inference for Distributed Multimodal Systems under Communication Delay Uncertainty
This addresses the challenge of robust real-time inference for connected cyber-physical systems, offering a novel approach to handle stochastic delays, though it appears incremental in improving existing non-blocking methods.
The paper tackled the problem of real-time inference in distributed multimodal systems under uncertain communication delays by proposing a neuro-inspired non-blocking inference paradigm with adaptive temporal windows of integration, achieving superior adaptability to network dynamics compared to state-of-the-art methods on the audio-visual event localization task.
Connected cyber-physical systems perform inference based on real-time inputs from multiple data streams. Uncertain communication delays across data streams challenge the temporal flow of the inference process. State-of-the-art (SotA) non-blocking inference methods rely on a reference-modality paradigm, requiring one modality input to be fully received before processing, while depending on costly offline profiling. We propose a novel, neuro-inspired non-blocking inference paradigm that primarily employs adaptive temporal windows of integration (TWIs) to dynamically adjust to stochastic delay patterns across heterogeneous streams while relaxing the reference-modality requirement. Our communication-delay-aware framework achieves robust real-time inference with finer-grained control over the accuracy-latency tradeoff. Experiments on the audio-visual event localization (AVEL) task demonstrate superior adaptability to network dynamics compared to SotA approaches.