Bo Peng

Jie He, Bo Peng, Yi Liao et al.

In order to deeply understand the capability of pretrained language models in text generation and conduct a diagnostic evaluation, we propose TGEA, an error-annotated dataset with multiple benchmark tasks for text generation from pretrained language models (PLMs). We use carefully selected prompt words to guide GPT-2 to generate candidate sentences, from which we select 47K for error annotation. Crowdsourced workers manually check each of these sentences and detect 12k erroneous sentences. We create an error taxonomy to cover 24 types of errors occurring in these erroneous sentences according to the nature of errors with respect to linguistics and knowledge (eg, common sense). For each erroneous span in PLM-generated sentences, we also detect another span that is closely associated with it. Each error is hence manually labeled with comprehensive annotations, including the span of the error, the associated span, minimal correction to the error, the type of the error, and rationale behind the error. Apart from the fully annotated dataset, we also present a detailed description of the data collection procedure, statistics and analysis of the dataset. This is the first dataset with comprehensive annotations for PLM-generated texts, which facilitates the diagnostic evaluation of PLM-based text generation. Furthermore, we use TGEA as a benchmark dataset and propose a series of automatic diagnosis tasks, including error detection, error type classification, associated span detection, error rationale generation, to further promote future study on the automatic error detection and correction on texts generated by pretrained language models.

Xinyi Ling, Hanwen Du, Bo Peng et al.

Leveraging multimodal data to drive breakthroughs in e-commerce applications through Multimodal Foundation Models (MFMs) is gaining increasing attention from the research community. However, there are significant challenges that hinder the optimal use of multimodal e-commerce data by foundation models: (1) the scarcity of large-scale, high-quality multimodal benchmark datasets; and (2) the lack of effective multimodal information integration methods. To address these challenges, in this paper, we introduce MMECInstruct, the first-ever, large-scale, and high-quality multimodal instruction dataset for e-commerce. We also develop CASLIE, a simple, lightweight, yet effective framework for integrating multimodal information for e-commerce. Leveraging MMECInstruct, we fine-tune a series of e-commerce MFMs within CASLIE, denoted as CASLIE models. Our comprehensive evaluation demonstrates that CASLIE models substantially outperform 5 categories of advanced baseline models in the in-domain evaluation. Moreover, CASLIE models show strong generalizability to out-of-domain settings. MMECInstruct and CASLIE models are publicly accessible through https://ninglab.github.io/CASLIE/.

Bo Peng, Renqiang Min, Xia Ning

Knowledge graph learning plays a critical role in integrating domain specific knowledge bases when deploying machine learning and data mining models in practice. Existing methods on knowledge graph learning primarily focus on modeling the relations among entities as translations among the relations and entities, and many of these methods are not able to handle zero-shot problems, when new entities emerge. In this paper, we present a new convolutional neural network (CNN)-based dual-chain model. Different from translation based methods, in our model, interactions among relations and entities are directly captured via CNN over their embeddings. Moreover, a secondary chain of learning is conducted simultaneously to incorporate additional information and to enable better performance. We also present an extension of this model, which incorporates descriptions of entities and learns a second set of entity embeddings from the descriptions. As a result, the extended model is able to effectively handle zero-shot problems. We conducted comprehensive experiments, comparing our methods with 15 methods on 8 benchmark datasets. Extensive experimental results demonstrate that our proposed methods achieve or outperform the state-of-the-art results on knowledge graph learning, and outperform other methods on zero-shot problems. In addition, our methods applied to real-world biomedical data are able to produce results that conform to expert domain knowledge.