PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
This addresses the problem of balancing safety metrics for reliable ML systems, offering a comprehensive solution that is not incremental but broadly applicable.
The paper tackles the challenge of improving multiple safety measures in machine learning systems without trade-offs, and introduces PixMix, a data augmentation strategy using complex pictures like fractals, which achieves near Pareto-optimal improvements across safety axes such as out-of-distribution robustness and anomaly detection.
In real-world applications of machine learning, reliable and safe systems must consider measures of performance beyond standard test set accuracy. These other goals include out-of-distribution (OOD) robustness, prediction consistency, resilience to adversaries, calibrated uncertainty estimates, and the ability to detect anomalous inputs. However, improving performance towards these goals is often a balancing act that today's methods cannot achieve without sacrificing performance on other safety axes. For instance, adversarial training improves adversarial robustness but sharply degrades other classifier performance metrics. Similarly, strong data augmentation and regularization techniques often improve OOD robustness but harm anomaly detection, raising the question of whether a Pareto improvement on all existing safety measures is possible. To meet this challenge, we design a new data augmentation strategy utilizing the natural structural complexity of pictures such as fractals, which outperforms numerous baselines, is near Pareto-optimal, and roundly improves safety measures.