TeamPath: Building MultiModal Pathology Experts with Reasoning AI Copilots
This addresses the challenge of building reliable AI copilots for real-world pathology scenarios, though it appears incremental as it builds on existing multimodal models with reinforcement learning enhancements.
The paper tackles the problem of AI models lacking rigorous reasoning and multi-task handling in computational pathology by introducing TeamPath, an AI system that assists pathologists in disease diagnosis and cross-modality integration, with human evaluations showing it can correct expert conclusions and improve efficiency.
Advances in AI have introduced several strong models in computational pathology to usher it into the era of multi-modal diagnosis, analysis, and interpretation. However, the current pathology-specific visual language models still lack capacities in making the diagnosis with rigorous reasoning paths as well as handling divergent tasks, and thus, challenges of building AI Copilots for real scenarios still exist. Here we introduce TeamPath, an AI system powered by reinforcement learning and router-enhanced solutions based on large-scale histopathology multimodal datasets, to work as a virtual assistant for expert-level disease diagnosis, patch-level information summarization, and cross-modality generation to integrate transcriptomic information for clinical usage. We also collaborate with pathologists from Yale School of Medicine to demonstrate that TeamPath can assist them in working more efficiently by identifying and correcting expert conclusions and reasoning paths. We also discuss the human evaluation results to support the reasoning quality from TeamPath. Overall, TeamPath can flexibly choose the best settings according to the needs, and serve as an innovative and reliable system for information communication across different modalities and experts.