Zhiqiang Wang

Zhiqiang Wang, Qingyun She, PengTao Zhang et al.

Click-through rate (CTR) estimation is a fundamental task in personalized advertising and recommender systems and it's important for ranking models to effectively capture complex high-order features.Inspired by the success of ELMO and Bert in NLP field, which dynamically refine word embedding according to the context sentence information where the word appears, we think it's also important to dynamically refine each feature's embedding layer by layer according to the context information contained in input instance in CTR estimation tasks. We can effectively capture the useful feature interactions for each feature in this way. In this paper, We propose a novel CTR Framework named ContextNet that implicitly models high-order feature interactions by dynamically refining each feature's embedding according to the input context. Specifically, ContextNet consists of two key components: contextual embedding module and ContextNet block. Contextual embedding module aggregates contextual information for each feature from input instance and ContextNet block maintains each feature's embedding layer by layer and dynamically refines its representation by merging contextual high-order interaction information into feature embedding. To make the framework specific, we also propose two models(ContextNet-PFFN and ContextNet-SFFN) under this framework by introducing linear contextual embedding network and two non-linear mapping sub-network in ContextNet block. We conduct extensive experiments on four real-world datasets and the experiment results demonstrate that our proposed ContextNet-PFFN and ContextNet-SFFN model outperform state-of-the-art models such as DeepFM and xDeepFM significantly.

Zhiqiang Wang, Qingyun She, Junlin Zhang

Click-Through Rate(CTR) estimation has become one of the most fundamental tasks in many real-world applications and it's important for ranking models to effectively capture complex high-order features. Shallow feed-forward network is widely used in many state-of-the-art DNN models such as FNN, DeepFM and xDeepFM to implicitly capture high-order feature interactions. However, some research has proved that addictive feature interaction, particular feed-forward neural networks, is inefficient in capturing common feature interaction. To resolve this problem, we introduce specific multiplicative operation into DNN ranking system by proposing instance-guided mask which performs element-wise product both on the feature embedding and feed-forward layers guided by input instance. We also turn the feed-forward layer in DNN model into a mixture of addictive and multiplicative feature interactions by proposing MaskBlock in this paper. MaskBlock combines the layer normalization, instance-guided mask, and feed-forward layer and it is a basic building block to be used to design new ranking model under various configurations. The model consisting of MaskBlock is called MaskNet in this paper and two new MaskNet models are proposed to show the effectiveness of MaskBlock as basic building block for composing high performance ranking systems. The experiment results on three real-world datasets demonstrate that our proposed MaskNet models outperform state-of-the-art models such as DeepFM and xDeepFM significantly, which implies MaskBlock is an effective basic building unit for composing new high performance ranking systems.

Tongwen Huang, Qingyun She, Zhiqiang Wang et al.

Advertising and feed ranking are essential to many Internet companies such as Facebook. Among many real-world advertising and feed ranking systems, click through rate (CTR) prediction plays a central role. In recent years, many neural network based CTR models have been proposed and achieved success such as Factorization-Machine Supported Neural Networks, DeepFM and xDeepFM. Many of them contain two commonly used components: embedding layer and MLP hidden layers. On the other side, gating mechanism is also widely applied in many research fields such as computer vision(CV) and natural language processing(NLP). Some research has proved that gating mechanism improves the trainability of non-convex deep neural networks. Inspired by these observations, we propose a novel model named GateNet which introduces either the feature embedding gate or the hidden gate to the embedding layer or hidden layers of DNN CTR models, respectively. The feature embedding gate provides a learnable feature gating module to select salient latent information from the feature-level. The hidden gate helps the model to implicitly capture the high-order interaction more effectively. Extensive experiments conducted on three real-world datasets demonstrate its effectiveness to boost the performance of various state-of-the-art models such as FM, DeepFM and xDeepFM on all datasets.

Xiaoming Chen, Weiqing You, Meng Jiao et al.

The braid group is an important non commutative group, at the same time, it is an important tool in quantum field theory with better topological structure, and often used as a research carrier for anti-quantum cryptographic algorithms. This paper proposed a difficult problem on a positive braid semi-group, and proved that the difficulty is not lower than the conjugate search problem. Based on this new difficult problem, we propose a new cryptosystem, which include a key exchange protocol and a public key encryption algorithm. Since our cryptosystem is implemented on a semi-group, it effectively avoids the analysis of attack algorithms on the cluster and makes our algorithm more secure.

Jianyi Zhang, Xiali Hei, Zhiqiang Wang

Crowdsourcing human-solving or online typing attacks are destructive problems. However, studies into these topics have been limited. In this paper, we focus on this kind of attacks whereby all the CAPTCHAs can be simply broken because of its design purpose. After pursuing a comprehensive analysis of the Typer phenomenon and the attacking mechanism of CAPTCHA, we present a new CAPTCHA design principle to distinguish human (Typer) from human (user). The core idea is that the challenge process of the CAPTCHA should contain the unique information with a private attribute. The notion of our idea is based on the information asymmetry between humans. Without this private information, Typers will not be able to finish the attack even if they recognize all the characters from the CAPTCHA. We formalize, design and implement two examples on our proposed principle, a character-based, and a datagram-based case, according to a web interaction and password handling program. We challenge the user to select the password from the random characters that are not in the password sequence or to place the randomly sorted sequences into the correct order. A novel generation algorithm with a fuzzy matching method has been proposed to add the capability of human error tolerance and the difficulty of random guess attack. Unlike other solutions, our approach does not need to modify the primary authentication protocol, user interface, and experience of the typical web service. The several user studies' results indicate that our proposed method is both simple (can be solved by humans accurately within less than 20 seconds) and efficient (the Typer can only deploy a random guess attack with a very low success rate).