Entropy Minimization vs. Diversity Maximization for Domain Adaptation
This addresses a key issue in domain adaptation for machine learning practitioners, though it appears incremental as it builds on existing entropy minimization approaches.
The paper tackles the problem of trivial solutions in unsupervised domain adaptation by proposing a method that balances entropy minimization with diversity maximization, achieving state-of-the-art performance on four datasets.
Entropy minimization has been widely used in unsupervised domain adaptation (UDA). However, existing works reveal that entropy minimization only may result into collapsed trivial solutions. In this paper, we propose to avoid trivial solutions by further introducing diversity maximization. In order to achieve the possible minimum target risk for UDA, we show that diversity maximization should be elaborately balanced with entropy minimization, the degree of which can be finely controlled with the use of deep embedded validation in an unsupervised manner. The proposed minimal-entropy diversity maximization (MEDM) can be directly implemented by stochastic gradient descent without use of adversarial learning. Empirical evidence demonstrates that MEDM outperforms the state-of-the-art methods on four popular domain adaptation datasets.