A Comparative Study of Demonstration Selection for Practical Large Language Models-based Next POI Prediction
It addresses demonstration selection for LLM-based POI prediction, offering practical insights for real-world applications, but is incremental as it compares existing methods.
This paper tackles the problem of predicting a user's next point-of-interest using large language models by evaluating demonstration selection strategies, finding that simpler heuristic methods like geographical proximity outperform complex embedding-based methods in accuracy and cost, sometimes exceeding fine-tuned models without training.
This paper investigates demonstration selection strategies for predicting a user's next point-of-interest (POI) using large language models (LLMs), aiming to accurately forecast a user's subsequent location based on historical check-in data. While in-context learning (ICL) with LLMs has recently gained attention as a promising alternative to traditional supervised approaches, the effectiveness of ICL significantly depends on the selected demonstration. Although previous studies have examined methods such as random selection, embedding-based selection, and task-specific selection, there remains a lack of comprehensive comparative analysis among these strategies. To bridge this gap and clarify the best practices for real-world applications, we comprehensively evaluate existing demonstration selection methods alongside simpler heuristic approaches such as geographical proximity, temporal ordering, and sequential patterns. Extensive experiments conducted on three real-world datasets indicate that these heuristic methods consistently outperform more complex and computationally demanding embedding-based methods, both in terms of computational cost and prediction accuracy. Notably, in certain scenarios, LLMs using demonstrations selected by these simpler heuristic methods even outperform existing fine-tuned models, without requiring further training. Our source code is available at: https://github.com/ryonsd/DS-LLM4POI.