AInsight: Augmenting Expert Decision-Making with On-the-Fly Insights Grounded in Historical Data
This work addresses the challenge for experts, such as doctors, in leveraging historical data in real-time conversations, though it is incremental as it builds on existing LLM and retrieval methods.
The paper tackled the problem of experts needing real-time access to historical data during decision-making conversations by developing a conversational interface that uses a retrieval-based LLM agent to generate insights from embedded datasets, showing effectiveness in simulated doctor-patient dialogues but also identifying challenges.
In decision-making conversations, experts must navigate complex choices and make on-the-spot decisions while engaged in conversation. Although extensive historical data often exists, the real-time nature of these scenarios makes it infeasible for decision-makers to review and leverage relevant information. This raises an interesting question: What if experts could utilize relevant past data in real-time decision-making through insights derived from past data? To explore this, we implemented a conversational user interface, taking doctor-patient interactions as an example use case. Our system continuously listens to the conversation, identifies patient problems and doctor-suggested solutions, and retrieves related data from an embedded dataset, generating concise insights using a pipeline built around a retrieval-based Large Language Model (LLM) agent. We evaluated the prototype by embedding Health Canada datasets into a vector database and conducting simulated studies using sample doctor-patient dialogues, showing effectiveness but also challenges, setting directions for the next steps of our work.