Xinyu Ma

Weiwei Sun, Lingyong Yan, Xinyu Ma et al. · baidu

Large Language Models (LLMs) have demonstrated remarkable zero-shot generalization across various language-related tasks, including search engines. However, existing work utilizes the generative ability of LLMs for Information Retrieval (IR) rather than direct passage ranking. The discrepancy between the pre-training objectives of LLMs and the ranking objective poses another challenge. In this paper, we first investigate generative LLMs such as ChatGPT and GPT-4 for relevance ranking in IR. Surprisingly, our experiments reveal that properly instructed LLMs can deliver competitive, even superior results to state-of-the-art supervised methods on popular IR benchmarks. Furthermore, to address concerns about data contamination of LLMs, we collect a new test set called NovelEval, based on the latest knowledge and aiming to verify the model's ability to rank unknown knowledge. Finally, to improve efficiency in real-world applications, we delve into the potential for distilling the ranking capabilities of ChatGPT into small specialized models using a permutation distillation scheme. Our evaluation results turn out that a distilled 440M model outperforms a 3B supervised model on the BEIR benchmark. The code to reproduce our results is available at www.github.com/sunnweiwei/RankGPT.

Weiwei Sun, Zheng Chen, Xinyu Ma et al. · baidu

Recent studies have demonstrated the great potential of Large Language Models (LLMs) serving as zero-shot relevance rankers. The typical approach involves making comparisons between pairs or lists of documents. Although effective, these listwise and pairwise methods are not efficient and also heavily rely on intricate prompt engineering. To tackle this problem, we introduce a novel instruction distillation method. The key idea is to distill the pairwise ranking ability of open-sourced LLMs to a simpler but more efficient pointwise ranking. Specifically, given the same LLM, we first rank documents using the effective pairwise approach with complex instructions, and then distill the teacher predictions to the pointwise approach with simpler instructions. Evaluation results on the BEIR, TREC, and ReDial datasets demonstrate that instruction distillation can improve efficiency by 10 to 100x and also enhance the ranking performance of LLMs. Furthermore, our approach surpasses the performance of existing supervised methods like monoT5 and is on par with the state-of-the-art zero-shot methods. The code to reproduce our results is available at www.github.com/sunnweiwei/RankGPT.

Liantao Ma, Chaohe Zhang, Junyi Gao et al.

Objective: Peritoneal Dialysis (PD) is one of the most widely used life-supporting therapies for patients with End-Stage Renal Disease (ESRD). Predicting mortality risk and identifying modifiable risk factors based on the Electronic Medical Records (EMR) collected along with the follow-up visits are of great importance for personalized medicine and early intervention. Here, our objective is to develop a deep learning model for a real-time, individualized, and interpretable mortality prediction model - AICare. Method and Materials: Our proposed model consists of a multi-channel feature extraction module and an adaptive feature importance recalibration module. AICare explicitly identifies the key features that strongly indicate the outcome prediction for each patient to build the health status embedding individually. This study has collected 13,091 clinical follow-up visits and demographic data of 656 PD patients. To verify the application universality, this study has also collected 4,789 visits of 1,363 hemodialysis dialysis (HD) as an additional experiment dataset to test the prediction performance, which will be discussed in the Appendix. Results: 1) Experiment results show that AICare achieves 81.6%/74.3% AUROC and 47.2%/32.5% AUPRC for the 1-year mortality prediction task on PD/HD dataset respectively, which outperforms the state-of-the-art comparative deep learning models. 2) This study first provides a comprehensive elucidation of the relationship between the causes of mortality in patients with PD and clinical features based on an end-to-end deep learning model. 3) This study first reveals the pattern of variation in the importance of each feature in the mortality prediction based on built-in interpretability. 4) We develop a practical AI-Doctor interaction system to visualize the trajectory of patients' health status and risk indicators.

Xinyu Ma, Xu Chu, Yasha Wang et al.

In healthcare prediction tasks, it is essential to exploit the correlations between medical features and learn better patient health representations. Existing methods try to estimate feature correlations only from data, or increase the quality of estimation by introducing task-specific medical knowledge. However, such methods either are difficult to estimate the feature correlations due to insufficient training samples, or cannot be generalized to other tasks due to reliance on specific knowledge. There are medical research revealing that not all the medical features are strongly correlated. Thus, to address the issues, we expect to group up strongly correlated features and learn feature correlations in a group-wise manner to reduce the learning complexity without losing generality. In this paper, we propose a general patient health representation learning framework MedFACT. We estimate correlations via measuring similarity between temporal patterns of medical features with kernel methods, and cluster features with strong correlations into groups. The feature group is further formulated as a correlation graph, and we employ graph convolutional networks to conduct group-wise feature interactions for better representation learning. Experiments on two real-world datasets demonstrate the superiority of MedFACT. The discovered medical findings are also confirmed by literature, providing valuable medical insights and explanations.

Xinyu Ma, Xu Chu, Zhibang Yang et al.

With the increasingly powerful performances and enormous scales of pretrained models, promoting parameter efficiency in fine-tuning has become a crucial need for effective and efficient adaptation to various downstream tasks. One representative line of fine-tuning methods is Orthogonal Fine-tuning (OFT), which rigorously preserves the angular distances within the parameter space to preserve the pretrained knowledge. Despite the empirical effectiveness, OFT still suffers low parameter efficiency at $\mathcal{O}(d^2)$ and limited capability of downstream adaptation. Inspired by Givens rotation, in this paper, we proposed quasi-Givens Orthogonal Fine-Tuning (qGOFT) to address the problems. We first use $\mathcal{O}(d)$ Givens rotations to accomplish arbitrary orthogonal transformation in $SO(d)$ with provable equivalence, reducing parameter complexity from $\mathcal{O}(d^2)$ to $\mathcal{O}(d)$. Then we introduce flexible norm and relative angular adjustments under soft orthogonality regularization to enhance the adaptation capability of downstream semantic deviations. Extensive experiments on various tasks and pretrained models validate the effectiveness of our methods.

Yixing Fan, Xiaohui Xie, Yinqiong Cai et al.

The core of information retrieval (IR) is to identify relevant information from large-scale resources and return it as a ranked list to respond to the user's information need. In recent years, the resurgence of deep learning has greatly advanced this field and leads to a hot topic named NeuIR (i.e., neural information retrieval), especially the paradigm of pre-training methods (PTMs). Owing to sophisticated pre-training objectives and huge model size, pre-trained models can learn universal language representations from massive textual data, which are beneficial to the ranking task of IR. Recently, a large number of works, which are dedicated to the application of PTMs in IR, have been introduced to promote the retrieval performance. Considering the rapid progress of this direction, this survey aims to provide a systematic review of pre-training methods in IR. To be specific, we present an overview of PTMs applied in different components of an IR system, including the retrieval component, the re-ranking component, and other components. In addition, we also introduce PTMs specifically designed for IR, and summarize available datasets as well as benchmark leaderboards. Moreover, we discuss some open challenges and highlight several promising directions, with the hope of inspiring and facilitating more works on these topics for future research.

Zhiyi Zhang, Tianyuan Yu, Xinyu Ma et al.

Recent years have witnessed the rapid deployment of smart homes; most of them are controlled by remote servers in the cloud. Such designs raise security and privacy concerns for end users. In this paper, we describe the design of Sovereign, a home IoT system framework that provides end users complete control of their home IoT systems. Sovereign lets home IoT devices and applications communicate via application-named data and secures data directly. This enables direct, secure, one-to-one and one-to-many device-to-device communication over wireless broadcast media. Sovereign utilizes semantic names to construct usable security solutions. We implement Sovereign as a publish-subscribe-based development platform together with a prototype home IoT controller. Our preliminary evaluation shows that Sovereign provides a systematic, easy-to-use solution to user-controlled, self-contained smart homes running on existing IoT hardware without imposing noticeable overhead.

Zhiyi Zhang, Yu Guan, Xinyu Ma et al.

As Personally Identifiable Information (PII) data sharing among multiple parties becomes increasingly common, so does the potential for data leakage. As required by new data protection regulations and laws, when PII leakage occurs, one must be able to reliably identify the leaking sources. Existing solutions utilize watermark technologies or data object allocation strategies to differentiate the data shared with different parties to identify potential leakers. However, these solutions lose their effectiveness under several attack scenarios, e.g., a data sender may leak the data and a receiver may deny the reception of certain shared data. Worse yet, multiple receivers might collude and apply a set of operations such as intersection, complement, and union to their received datasets before leaking them, making the task of leaker identification even more difficult. In this paper, we propose AuditShare, a PII dataset sharing system with reliable leaking source identification. Firstly, taking advantage of the intrinsic properties of PII data, AuditShare allocates data objects to individual sharing parties by PII attributes. Secondly, AuditShare obliviously transfers data between the sender and each receiver and uses a Merkle Tree as an immutable record of the sharing. Thirdly, a knowledge-based identification algorithm is proposed to identify a guilty sender or colluding/non-colluding receivers. Through our evaluation, we show that: (i) With a modest amount of leaked data, AuditShare can accurately (accuracy>99.99%) and undeniably identify all the guilty parties in different cases; (ii) It only takes 0.5 second to share 100,000 data objects in AuditShare, which is practical in real-world deployment.

Zhiyi Zhang, Vishrant Vasavada, Xinyu Ma et al.

With the ever growing Internet of Things (IoT) market, ledger systems are facing new challenges to efficiently store and secure enormous customer records collected by the IoT devices. The authenticity, availability, and integrity of these records are critically important for both business providers and customers. In this paper, we describe DLedger, a lightweight and resilient distributed ledger system. Instead of a single chain of blocks, DLedger builds the ledger over a directed acyclic graph (DAG), so that its operations can tolerate network partition and intermittent connectivity. Instead of compute-intensive Proof-of-Work (PoW), DLedger utilizes Proof-of-Authentication (PoA), whose light-weight operations are IoT-friendly, to achieve consensus. Furthermore, DLedger is built upon a data-centric network called Named Data Networking (NDN), which facilitates the peer-to-peer data dissemination in heterogeneous IoT networks.