70.8HCMay 13
Discovery-Oriented Faceting: From Coverage to Blind-Spot DiscoveryYoudi Li
When people explore large document collections to build understanding, they face a challenge: existing AI tools help them see what is central but tend to hide what is unusual. Summarization and topic modeling optimize for coverage, representing main themes while pushing minority viewpoints and edge cases out of view. This matters because discovery often depends on noticing what does not fit, such as unexpected findings, minority positions, or gaps in the literature. When tools hide this content, users may miss insights that could change their understanding. In this paper, we explore an alternative objective: blind-spot discovery, where the goal is to surface content that coverage methods suppress so that people can judge its significance for themselves. We propose three design goals and illustrate them through DOF (Discovery-Oriented Faceting), a system that organizes documents into categories with explicit boundaries, ranks categories by distinctiveness rather than size, and supports iterative refinement. Comparing DOF against coverage-based ranking across four domains, we find that the two approaches surface fundamentally different content, with DOF promoting specialized categories that coverage methods bury. We discuss how shifting from coverage to discovery may offer a complementary mode of support for people exploring large text collections.
CLApr 11, 2024
ODA: Observation-Driven Agent for integrating LLMs and Knowledge GraphsLei Sun, Zhengwei Tao, Youdi Li et al.
The integration of Large Language Models (LLMs) and knowledge graphs (KGs) has achieved remarkable success in various natural language processing tasks. However, existing methodologies that integrate LLMs and KGs often navigate the task-solving process solely based on the LLM's analysis of the question, overlooking the rich cognitive potential inherent in the vast knowledge encapsulated in KGs. To address this, we introduce Observation-Driven Agent (ODA), a novel AI agent framework tailored for tasks involving KGs. ODA incorporates KG reasoning abilities via global observation, which enhances reasoning capabilities through a cyclical paradigm of observation, action, and reflection. Confronting the exponential explosion of knowledge during observation, we innovatively design a recursive observation mechanism. Subsequently, we integrate the observed knowledge into the action and reflection modules. Through extensive experiments, ODA demonstrates state-of-the-art performance on several datasets, notably achieving accuracy improvements of 12.87% and 8.9%.
CLOct 16, 2024
Pyramid-Driven Alignment: Pyramid Principle Guided Integration of Large Language Models and Knowledge GraphsLei Sun, Xinchen Wang, Youdi Li
Large Language Models (LLMs) possess impressive reasoning abilities but are prone to generating incorrect information, often referred to as hallucinations. While incorporating external Knowledge Graphs (KGs) can partially mitigate this issue, existing methods primarily treat KGs as static knowledge repositories, overlooking the critical disparity between KG and LLM knowledge, and failing to fully exploit the reasoning capabilities inherent in KGs. To address these limitations, we propose Pyramid-Driven Alignment (PDA), a novel framework for seamlessly integrating LLMs with KGs. PDA utilizes Pyramid Principle analysis to construct a hierarchical pyramid structure. This structure is designed to reflect the input question and generate more validated deductive knowledge, thereby enhancing the alignment of LLMs and KGs and ensuring more cohesive integration. Furthermore, PDA employs a recursive mechanism to harness the underlying reasoning abilities of KGs, resulting in more accurate knowledge retrieval for question-answering tasks. Our experimental results reveal a substantial performance advantage of PDA over state-of-the-art baselines, with improvements reaching 26.70% and 26.78%.