Distributionally Robust Set Representation Learning Under Inference-Time Element Corruption
For practitioners deploying set representation models in noisy environments, this work provides a method to enhance robustness against element corruption, though the improvements are incremental over existing robust learning approaches.
The paper tackles inference-time element corruption in set representation learning, proposing SW-DRSO, a distributionally robust optimization framework that improves robustness against element-level degradations while maintaining high performance across four tasks.
Standard Set Representation Learning methods typically excel on curated data but often overlook the challenge of inference-time element corruption. This refers to scenarios where deployed models encounter element-level degradations, such as outliers or missing components, that may distort set representation and degrade performance. We propose SW-DRSO, a distributionally robust optimization framework tailored for sets. Rather than minimizing loss solely on observed training data, SW-DRSO optimizes a tractable surrogate of the worst-case expected loss over a family of plausible inference-time variations. We introduce a barycentric adversary that approximates the intractable search over corrupted sets by a differentiable training-time optimization over simplex weights. Extensive experiments across four tasks demonstrate that SW-DRSO effectively enhances robustness against corruption while maintaining high overall performance.