Hongyang Xue

Minghao Chen, Hongyang Xue, Deng Cai

Deep neural networks for semantic segmentation always require a large number of samples with pixel-level labels, which becomes the major difficulty in their real-world applications. To reduce the labeling cost, unsupervised domain adaptation (UDA) approaches are proposed to transfer knowledge from labeled synthesized datasets to unlabeled real-world datasets. Recently, some semi-supervised learning methods have been applied to UDA and achieved state-of-the-art performance. One of the most popular approaches in semi-supervised learning is the entropy minimization method. However, when applying the entropy minimization to UDA for semantic segmentation, the gradient of the entropy is biased towards samples that are easy to transfer. To balance the gradient of well-classified target samples, we propose the maximum squares loss. Our maximum squares loss prevents the training process being dominated by easy-to-transfer samples in the target domain. Besides, we introduce the image-wise weighting ratio to alleviate the class imbalance in the unlabeled target domain. Both synthetic-to-real and cross-city adaptation experiments demonstrate the effectiveness of our proposed approach. The code is released at https://github. com/ZJULearning/MaxSquareLoss.

Hongyang Xue, Zhou Zhao, Deng Cai

A number of visual question answering approaches have been proposed recently, aiming at understanding the visual scenes by answering the natural language questions. While the image question answering has drawn significant attention, video question answering is largely unexplored. Video-QA is different from Image-QA since the information and the events are scattered among multiple frames. In order to better utilize the temporal structure of the videos and the phrasal structures of the answers, we propose two mechanisms: the re-watching and the re-reading mechanisms and combine them into the forgettable-watcher model. Then we propose a TGIF-QA dataset for video question answering with the help of automatic question generation. Finally, we evaluate the models on our dataset. The experimental results show the effectiveness of our proposed models.

Hongyang Xue, Deng Cai

In [18], Mozerov et al. propose to perform stereo matching as a two-step energy minimization problem. For the first step they solve a fully connected MRF model. And in the next step the marginal output is employed as the unary cost for a locally connected MRF model. In this paper we intend to combine the two steps of energy minimization in order to improve stereo matching results. We observe that the fully connected MRF leads to smoother disparity maps, while the locally connected MRF achieves superior results in fine-structured regions. Thus we propose to jointly solve the fully connected and locally connected models, taking both their advantages into account. The joint model is solved by mean field approximations. While remaining efficient, our joint model outperforms the two-step energy minimization approach in both time and estimation error on the Middlebury stereo benchmark v3.