Benchmarking drug-drug interaction prediction methods: a perspective of distribution changes
This work addresses the challenge of realistic evaluation for drug-drug interaction prediction, which is crucial for new drug development, by providing a benchmark that simulates distribution changes, though it is incremental as it builds on existing methods with a new evaluation framework.
The authors tackled the problem of emerging drug-drug interaction (DDI) prediction under distribution changes by proposing DDI-Ben, a benchmarking framework that simulates these changes and evaluates methods, showing that most existing approaches suffer substantial performance degradation while LLM-based methods and drug-related textual information offer promising robustness.
Motivation: Emerging drug-drug interaction (DDI) prediction is crucial for new drugs but is hindered by distribution changes between known and new drugs in real-world scenarios. Current evaluation often neglects these changes, relying on unrealistic i.i.d. split due to the absence of drug approval data. Results: We propose DDI-Ben, a benchmarking framework for emerging DDI prediction under distribution changes. DDI-Ben introduces a distribution change simulation framework that leverages distribution changes between drug sets as a surrogate for real-world distribution changes of DDIs, and is compatible with various drug split strategies. Through extensive benchmarking on ten representative methods, we show that most existing approaches suffer substantial performance degradation under distribution changes. Our analysis further indicates that large language model (LLM) based methods and the integration of drug-related textual information offer promising robustness against such degradation. To support future research, we release the benchmark datasets with simulated distribution changes. Overall, DDI-Ben highlights the importance of explicitly addressing distribution changes and provides a foundation for developing more resilient methods for emerging DDI prediction. Availability and implementation: Our code and data are available at https://github.com/LARS-research/DDI-Bench.