AlgoPilot: Fully Autonomous Program Synthesis Without Human-Written Programs
This work establishes a new paradigm for algorithm discovery, potentially impacting fields like automated software development and AI research, though it is foundational and not yet applied broadly.
The paper tackles the problem of program synthesis by introducing AlgoPilot, a method that generates algorithms from scratch without human-written programs, using reinforcement learning guided by a Trajectory Language Model, and demonstrates it can produce interpretable algorithms like Bubble Sort for sorting tasks.
Program synthesis has traditionally relied on human-provided specifications, examples, or prior knowledge to generate functional algorithms. Existing methods either emulate human-written algorithms or solve specific tasks without generating reusable programmatic logic, limiting their ability to create novel algorithms. We introduce AlgoPilot, a groundbreaking approach for fully automated program synthesis without human-written programs or trajectories. AlgoPilot leverages reinforcement learning (RL) guided by a Trajectory Language Model (TLM) to synthesize algorithms from scratch. The TLM, trained on trajectories generated by random Python functions, serves as a soft constraint during the RL process, aligning generated sequences with patterns likely to represent valid algorithms. Using sorting as a test case, AlgoPilot demonstrates its ability to generate trajectories that are interpretable as classical algorithms, such as Bubble Sort, while operating without prior algorithmic knowledge. This work establishes a new paradigm for algorithm discovery and lays the groundwork for future advancements in autonomous program synthesis.