DeepCodeSeek: Real-Time API Retrieval for Context-Aware Code Generation
This work addresses API retrieval for enterprise-specific code generation, offering a practical solution with improved accuracy and efficiency, though it appears incremental in its approach.
The paper tackles the problem of API retrieval for code generation by introducing a new dataset from real-world ServiceNow Script Includes to address API leaks in benchmarks, achieving 87.86% top-40 retrieval accuracy. It also develops a post-training pipeline for a compact 0.6B reranker that outperforms an 8B model with 2.5x reduced latency.
Current search techniques are limited to standard RAG query-document applications. In this paper, we propose a novel technique to expand the code and index for predicting the required APIs, directly enabling high-quality, end-to-end code generation for auto-completion and agentic AI applications. We address the problem of API leaks in current code-to-code benchmark datasets by introducing a new dataset built from real-world ServiceNow Script Includes that capture the challenge of unclear API usage intent in the code. Our evaluation metrics show that this method achieves 87.86% top-40 retrieval accuracy, allowing the critical context with APIs needed for successful downstream code generation. To enable real-time predictions, we develop a comprehensive post-training pipeline that optimizes a compact 0.6B reranker through synthetic dataset generation, supervised fine-tuning, and reinforcement learning. This approach enables our compact reranker to outperform a much larger 8B model while maintaining 2.5x reduced latency, effectively addressing the nuances of enterprise-specific code without the computational overhead of larger models.