AI-assisted Coding with Cody: Lessons from Context Retrieval and Evaluation for Code Recommendations
It addresses the problem of improving AI-assisted coding tools for developers, but it is incremental as it focuses on lessons and connections rather than new breakthroughs.
The paper analyzes LLM-based coding assistants, connecting them to traditional recommender systems and highlighting the importance of providing relevant context to LLMs for code recommendations, with lessons from context enhancements and evaluation methods.
In this work, we discuss a recently popular type of recommender system: an LLM-based coding assistant. Connecting the task of providing code recommendations in multiple formats to traditional RecSys challenges, we outline several similarities and differences due to domain specifics. We emphasize the importance of providing relevant context to an LLM for this use case and discuss lessons learned from context enhancements & offline and online evaluation of such AI-assisted coding systems.