Order-preserving Consistency Regularization for Domain Adaptation and Generalization
This work addresses domain adaptation and generalization challenges for AI systems, but it is incremental as it builds on existing consistency regularization methods.
The paper tackles the problem of deep learning models being oversensitive to domain-specific attributes in cross-domain tasks by proposing Order-preserving Consistency Regularization (OCR), which enforces order-preserving properties in predictions to improve robustness, achieving clear advantages on five different cross-domain tasks.
Deep learning models fail on cross-domain challenges if the model is oversensitive to domain-specific attributes, e.g., lightning, background, camera angle, etc. To alleviate this problem, data augmentation coupled with consistency regularization are commonly adopted to make the model less sensitive to domain-specific attributes. Consistency regularization enforces the model to output the same representation or prediction for two views of one image. These constraints, however, are either too strict or not order-preserving for the classification probabilities. In this work, we propose the Order-preserving Consistency Regularization (OCR) for cross-domain tasks. The order-preserving property for the prediction makes the model robust to task-irrelevant transformations. As a result, the model becomes less sensitive to the domain-specific attributes. The comprehensive experiments show that our method achieves clear advantages on five different cross-domain tasks.