Descriptive History Representations: Learning Representations by Answering Questions
This work addresses the challenge of summarizing histories for optimal control in AI systems, with applications in user modeling, but it appears incremental as it builds on existing representation learning concepts.
The paper tackles the problem of compressing long interaction histories for decision-making in partially observable environments by introducing Descriptive History Representations (DHRs), which are sufficient statistics optimized to answer relevant questions about past interactions and future outcomes, validated on user modeling tasks with public movie and shopping datasets to generate interpretable textual user profiles.
Effective decision making in partially observable environments requires compressing long interaction histories into informative representations. We introduce Descriptive History Representations (DHRs): sufficient statistics characterized by their capacity to answer relevant questions about past interactions and potential future outcomes. DHRs focus on capturing the information necessary to address task-relevant queries, providing a structured way to summarize a history for optimal control. We propose a multi-agent learning framework, involving representation, decision, and question-asking components, optimized using a joint objective that balances reward maximization with the representation's ability to answer informative questions. This yields representations that capture the salient historical details and predictive structures needed for effective decision making. We validate our approach on user modeling tasks with public movie and shopping datasets, generating interpretable textual user profiles which serve as sufficient statistics for predicting preference-driven behavior of users.