Learning Text-Image Joint Embedding for Efficient Cross-Modal Retrieval with Deep Feature EngineeringZhongwei Xie, Ling Liu, Yanzhao Wu et al.
This paper introduces a two-phase deep feature engineering framework for efficient learning of semantics enhanced joint embedding, which clearly separates the deep feature engineering in data preprocessing from training the text-image joint embedding model. We use the Recipe1M dataset for the technical description and empirical validation. In preprocessing, we perform deep feature engineering by combining deep feature engineering with semantic context features derived from raw text-image input data. We leverage LSTM to identify key terms, deep NLP models from the BERT family, TextRank, or TF-IDF to produce ranking scores for key terms before generating the vector representation for each key term by using word2vec. We leverage wideResNet50 and word2vec to extract and encode the image category semantics of food images to help semantic alignment of the learned recipe and image embeddings in the joint latent space. In joint embedding learning, we perform deep feature engineering by optimizing the batch-hard triplet loss function with soft-margin and double negative sampling, taking into account also the category-based alignment loss and discriminator-based alignment loss. Extensive experiments demonstrate that our SEJE approach with deep feature engineering significantly outperforms the state-of-the-art approaches.
0.2CLOct 9, 2018
Using Sentiment Representation Learning to Enhance Gender Classification for User ProfilingYunpei Zheng, Lin Li, Luo Zhong et al.
User profiling means exploiting the technology of machine learning to predict attributes of users, such as demographic attributes, hobby attributes, preference attributes, etc. It's a powerful data support of precision marketing. Existing methods mainly study network behavior, personal preferences, post texts to build user profile. Through our data analysis of micro-blog, we find that females show more positive and have richer emotions than males in online social platform. This difference is very conducive to the distinction between genders. Therefore, we argue that sentiment context is important as well for user profiling.This paper focuses on exploiting microblog user posts to predict one of the demographic labels: gender. We propose a Sentiment Representation Learning based Multi-Layer Perceptron(SRL-MLP) model to classify gender. First we build a sentiment polarity classifier in advance by training Long Short-Term Memory(LSTM) model on e-commerce review corpus. Next we transfer sentiment representation to a basic MLP network. Last we conduct experiments on gender classification by sentiment representation. Experimental results show that our approach can improve gender classification accuracy by 5.53\%, from 84.20\% to 89.73\%.
0.9CVOct 4, 2018
Image-to-Video Person Re-Identification by Reusing Cross-modal EmbeddingsZhongwei Xie, Lin Li, Xian Zhong et al.
Image-to-video person re-identification identifies a target person by a probe image from quantities of pedestrian videos captured by non-overlapping cameras. Despite the great progress achieved,it's still challenging to match in the multimodal scenario,i.e. between image and video. Currently,state-of-the-art approaches mainly focus on the task-specific data,neglecting the extra information on the different but related tasks. In this paper,we propose an end-to-end neural network framework for image-to-video person reidentification by leveraging cross-modal embeddings learned from extra information.Concretely speaking,cross-modal embeddings from image captioning and video captioning models are reused to help learned features be projected into a coordinated space,where similarity can be directly computed. Besides,training steps from fixed model reuse approach are integrated into our framework,which can incorporate beneficial information and eventually make the target networks independent of existing models. Apart from that,our proposed framework resorts to CNNs and LSTMs for extracting visual and spatiotemporal features,and combines the strengths of identification and verification model to improve the discriminative ability of the learned feature. The experimental results demonstrate the effectiveness of our framework on narrowing down the gap between heterogeneous data and obtaining observable improvement in image-to-video person re-identification.
3.1CRFeb 19, 2016
On the Nonexistence of the Ding-Helleseth-Martinsens Constructions of Almost Difference Set for Cyclotomic Classes of Order 6Minglong Qi, Shengwu Xiong, Jinbgling Yuan et al.
Pseudorandom sequences with optimal three-level autocorrelation have important applications in CDMA communication systems. Constructing the sequences with three-level autocorrelation is equivalent to finding cyclic almost difference sets as their supports. In a paper of Ding, Helleseth, and Martinsen, the authors developed a new method known as the Ding-Helleseth-Martinsens Constructions in literature to construct the almost difference set using product set between GF(2) and union sets of cyclotomic classes of order 4. In this correspondence, we show that there do not exist such constructions for cyclotomic classes of order 6.
3.2CRDec 12, 2015
On the Linear Complexity of Generalized Cyclotomic Quaternary Sequences with Length $2pq$Minglong Qi, Shengwu Xiong, Jingling Yuan et al.
In this paper, the linear complexity over $\mathbf{GF}(r)$ of generalized cyclotomic quaternary sequences with period $2pq$ is determined, where $ r $ is an odd prime such that $r \ge 5$ and $r\notin \lbrace p,q\rbrace$. The minimal value of the linear complexity is equal to $\tfrac{5pq+p+q+1}{4}$ which is greater than the half of the period $2pq$. According to the Berlekamp-Massey algorithm, these sequences are viewed as enough good for the use in cryptography. We show also that if the character of the extension field $\mathbf{GF}(r^{m})$, $r$, is chosen so that $\bigl(\tfrac{r}{p}\bigr) = \bigl(\tfrac{r}{q}\bigr) = -1$, $r\nmid 3pq-1$, and $r\nmid 2pq-4$, then the linear complexity can reach the maximal value equal to the length of the sequences.