Donghui Wang

Pengyang Li, Donghui Wang

In this paper, we propose a multi-stage and high-resolution model for image synthesis that uses fine-grained attributes and masks as input. With a fine-grained attribute, the proposed model can detailedly constrain the features of the generated image through rich and fine-grained semantic information in the attribute. With mask as prior, the model in this paper is constrained so that the generated images conform to visual senses, which will reduce the unexpected diversity of samples generated from the generative adversarial network. This paper also proposes a scheme to improve the discriminator of the generative adversarial network by simultaneously discriminating the total image and sub-regions of the image. In addition, we propose a method for optimizing the labeled attribute in datasets, which reduces the manual labeling noise. Extensive quantitative results show that our image synthesis model generates more realistic images.

Shengyu Zhang, Donghui Wang, Zhou Zhao et al.

In this paper, we investigate the problem of text-to-pedestrian synthesis, which has many potential applications in art, design, and video surveillance. Existing methods for text-to-bird/flower synthesis are still far from solving this fine-grained image generation problem, due to the complex structure and heterogeneous appearance that the pedestrians naturally take on. To this end, we propose the Multi-Grained Discrimination enhanced Generative Adversarial Network, that capitalizes a human-part-based Discriminator (HPD) and a self-cross-attended (SCA) global Discriminator in order to capture the coherence of the complex body structure. A fined-grained word-level attention mechanism is employed in the HPD module to enforce diversified appearance and vivid details. In addition, two pedestrian generation metrics, named Pose Score and Pose Variance, are devised to evaluate the generation quality and diversity, respectively. We conduct extensive experiments and ablation studies on the caption-annotated pedestrian dataset, CUHK Person Description Dataset. The substantial improvement over the various metrics demonstrates the efficacy of MGD-GAN on the text-to-pedestrian synthesis scenario.

Yuetan Lin, Zhangyang Pang, Donghui Wang et al.

Visual question answering (VQA) has witnessed great progress since May, 2015 as a classic problem unifying visual and textual data into a system. Many enlightening VQA works explore deep into the image and question encodings and fusing methods, of which attention is the most effective and infusive mechanism. Current attention based methods focus on adequate fusion of visual and textual features, but lack the attention to where people focus to ask questions about the image. Traditional attention based methods attach a single value to the feature at each spatial location, which losses many useful information. To remedy these problems, we propose a general method to perform saliency-like pre-selection on overlapped region features by the interrelation of bidirectional LSTM (BiLSTM), and use a novel element-wise multiplication based attention method to capture more competent correlation information between visual and textual features. We conduct experiments on the large-scale COCO-VQA dataset and analyze the effectiveness of our model demonstrated by strong empirical results.

Shu Kong, Donghui Wang

This note presents some representative methods which are based on dictionary learning (DL) for classification. We do not review the sophisticated methods or frameworks that involve DL for classification, such as online DL and spatial pyramid matching (SPM), but rather, we concentrate on the direct DL-based classification methods. Here, the "so-called direct DL-based method" is the approach directly deals with DL framework by adding some meaningful penalty terms. By listing some representative methods, we can roughly divide them into two categories, i.e. (1) directly making the dictionary discriminative and (2) forcing the sparse coefficients discriminative to push the discrimination power of the dictionary. From this taxonomy, we can expect some extensions of them as future researches.

Shu Kong, Donghui Wang

In practical applications, we often have to deal with high order data, such as a grayscale image and a video sequence are intrinsically 2nd-order tensor and 3rd-order tensor, respectively. For doing clustering or classification of these high order data, it is a conventional way to vectorize these data before hand, as PCA or FDA does, which often induce the curse of dimensionality problem. For this reason, experts have developed many methods to deal with the tensorial data, such as multilinear PCA, multilinear LDA, and so on. In this paper, we still address the problem of high order data representation and recognition, and propose to study the result of merging multilinear PCA and multilinear LDA into one scenario, we name it \textbf{GDA} for the abbreviation of Generalized Discriminant Analysis. To evaluate GDA, we perform a series of experiments, and the experimental results demonstrate our GDA outperforms a selection of competing methods such (2D)$^2$PCA, (2D)$^2$LDA, and MDA.

Shu Kong, Donghui Wang

This paper introduces a simple but highly efficient ensemble for robust texture classification, which can effectively deal with translation, scale and changes of significant viewpoint problems. The proposed method first inherits the spirit of spatial pyramid matching model (SPM), which is popular for encoding spatial distribution of local features, but in a flexible way, partitioning the original image into different levels and incorporating different overlapping patterns of each level. This flexible setup helps capture the informative features and produces sufficient local feature codes by some well-chosen aggregation statistics or pooling operations within each partitioned region, even when only a few sample images are available for training. Then each texture image is represented by several orderless feature codes and thereby all the training data form a reliable feature pond. Finally, to take full advantage of this feature pond, we develop a collaborative representation-based strategy with locality constraint (LC-CRC) for the final classification, and experimental results on three well-known public texture datasets demonstrate the proposed approach is very competitive and even outperforms several state-of-the-art methods. Particularly, when only a few samples of each category are available for training, our approach still achieves very high classification performance.