Ultra Strong Machine Learning: Teaching Humans Active Learning Strategies via Automated AI Explanations
This work addresses the challenge of making symbolic learning systems teachable to humans, though it appears incremental as it builds on prior USML approaches by replacing hand-crafted templates with automated generation.
The authors tackled the problem of automating explanations for machine-learned logic programs by developing LENS, a neuro-symbolic method that combines symbolic program synthesis with large language models, which generated superior explanations compared to existing methods. However, in human learning experiments across three domains, the results showed no significant performance improvements, indicating that comprehensive LLM responses may overwhelm users rather than aid learning.
Ultra Strong Machine Learning (USML) refers to symbolic learning systems that not only improve their own performance but can also teach their acquired knowledge to quantifiably improve human performance. In this work, we present LENS (Logic Programming Explanation via Neural Summarisation), a neuro-symbolic method that combines symbolic program synthesis with large language models (LLMs) to automate the explanation of machine-learned logic programs in natural language. LENS addresses a key limitation of prior USML approaches by replacing hand-crafted explanation templates with scalable automated generation. Through systematic evaluation using multiple LLM judges and human validation, we demonstrate that LENS generates superior explanations compared to direct LLM prompting and hand-crafted templates. To investigate whether LENS can teach transferable active learning strategies, we carried out a human learning experiment across three related domains. Our results show no significant human performance improvements, suggesting that comprehensive LLM responses may overwhelm users for simpler problems rather than providing learning support. Our work provides a solid foundation for building effective USML systems to support human learning. The source code is available on: https://github.com/lun-ai/LENS.git.