LEKA:LLM-Enhanced Knowledge Augmentation
This work addresses the problem of enabling models to autonomously retrieve and transfer knowledge, which is incremental as it builds on existing transfer learning methods by enhancing knowledge source identification.
The paper tackles the challenge of teaching models to identify appropriate knowledge sources for transfer learning, proposing LEKA, a knowledge augmentation method that actively searches for and harmonizes external knowledge with target domain data, resulting in significant improvements in computational cost reduction, data alignment automation, and transfer learning outcomes.
Humans excel in analogical learning and knowledge transfer and, more importantly, possess a unique understanding of identifying appropriate sources of knowledge. From a model's perspective, this presents an interesting challenge. If models could autonomously retrieve knowledge useful for transfer or decision-making to solve problems, they would transition from passively acquiring to actively accessing and learning from knowledge. However, filling models with knowledge is relatively straightforward -- it simply requires more training and accessible knowledge bases. The more complex task is teaching models about which knowledge can be analogized and transferred. Therefore, we design a knowledge augmentation method, LEKA, for knowledge transfer that actively searches for suitable knowledge sources that can enrich the target domain's knowledge. This LEKA method extracts key information from the target domain's textual information, retrieves pertinent data from external data libraries, and harmonizes retrieved data with the target domain data in feature space and marginal probability measures. We validate the effectiveness of our approach through extensive experiments across various domains and demonstrate significant improvements over traditional methods in reducing computational costs, automating data alignment, and optimizing transfer learning outcomes.