Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing
It addresses the problem of certifying robustness in multimodal models against heterogeneous perturbations, which is critical for safety-critical applications like text-image filtering.
The paper introduces a unified randomized smoothing framework for mixed discrete-continuous inputs, providing the first model-agnostic Neyman-Pearson certificate for joint perturbations in multimodal safety filtering.
Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in isolation. This limitation becomes critical in multimodal models, where decisions depend on cross-modal semantics and adversaries can jointly perturb heterogeneous inputs, rendering unimodal certificates insufficient. We introduce a unified randomized smoothing framework for mixed discrete--continuous inputs based on an analytically tractable Neyman--Pearson formulation of the joint worst-case problem. By analyzing the joint likelihood ordering induced by factorized discrete and continuous noise, our approach yields a closed-form, one-dimensional certificate that strictly generalizes both Gaussian (image-only) and discrete (text-only) randomized smoothing. We validate the framework on multimodal safety filtering, providing, to our knowledge, the first model-agnostic Neyman--Pearson certificate for joint discrete-token and continuous-image perturbations in interaction-dependent text--image safety filtering.