Chunhui Ai

Chunhui Ai, Derui Wang, Xu Yan et al.

With so many articles of varying qualities being produced every moment, it is a very urgent task to screen outstanding articles and commit them to social media. To our best knowledge, there is a lack of datasets and mature research works in identifying high-quality articles. Consequently, we conduct some surveys and finalize 7 objective indicators to annotate the quality of 10k articles. During annotation, we find that many characteristics of high-quality articles (e.g., background) rely more on extensive external knowledge than inner semantic information of articles. In response, we link extracted article entities to Baidu Encyclopedia, then propose Knowledge Enhanced article Quality Identification (KE-QI) dataset. To make better use of external knowledge, we propose a compound model which fuses the text and external knowledge information via a gate unit to classify the quality of an article. Our experimental results on KE-QI show that with initialization of our pre-trained Node2Vec model, our model achieves about 78\% $F_1$, outperforming other baselines.

Jun Gao, Ziqiang Cao, Shaoyao Huang et al.

ChatGPT is instruct-tuned to generate general and human-expected content to align with human preference through Reinforcement Learning from Human Feedback (RLHF), meanwhile resulting in generated responses not salient enough. Therefore, in this case, ChatGPT may fail to satisfy domain requirements in zero-shot settings, leading to poor ROUGE scores. Inspired by the In-Context Learning (ICL) and retelling ability of ChatGPT, this paper proposes PADS, a \textbf{P}ipeline for \textbf{A}ssisting ChatGPT in \textbf{D}omain \textbf{S}ummarization. PADS consists of a retriever to retrieve similar examples from corpora and a rank model to rerank the multiple candidate summaries generated by ChatGPT. Specifically, given an inference document, we first retrieve an in-context demonstration via the retriever. Then, we require ChatGPT to generate $k$ candidate summaries for the inference document at a time under the guidance of the retrieved demonstration. Finally, the rank model independently scores the $k$ candidate summaries according to their quality and selects the optimal one. We extensively explore dense and sparse retrieval methods to select effective demonstrations for reference and efficiently train the rank model to reflect the quality of candidate summaries for each given summarized document. Additionally, PADS contains merely 400M trainable parameters originating from the rank model and we merely collect 2.5k data to train it. We evaluate PADS on five datasets from different domains, and the result indicates that each module in PADS is committed to effectively guiding ChatGPT to generate salient summaries fitting different domain requirements. Specifically, in the popular summarization dataset Gigaword, PADS achieves over +8 gain on ROUGE-L, compared with the naive ChatGPT in the zero-shot setting. \footnote{Our code are available at \url{https://github.com/jungao1106/PADS}}