A Layered Architecture for Developing and Enhancing Capabilities in Large Language Model-based Software Systems
This work provides developers with actionable insights for selecting technologies in LLM-based software systems, promoting robustness and scalability, but it is incremental as it builds on existing methods without introducing new paradigms.
The paper tackles the challenge of systematically developing and enhancing capabilities in large language model-based software systems by introducing a layered architecture that organizes development into distinct layers, and demonstrates its utility through practical case studies.
Significant efforts has been made to expand the use of Large Language Models (LLMs) beyond basic language tasks. While the generalizability and versatility of LLMs have enabled widespread adoption, evolving demands in application development often exceed their native capabilities. Meeting these demands may involve a diverse set of methods, such as enhancing creativity through either inference temperature adjustments or creativity-provoking prompts. Selecting the right approach is critical, as different methods lead to trade-offs in engineering complexity, scalability, and operational costs. This paper introduces a layered architecture that organizes LLM software system development into distinct layers, each characterized by specific attributes. By aligning capabilities with these layers, the framework encourages the systematic implementation of capabilities in effective and efficient ways that ultimately supports desired functionalities and qualities. Through practical case studies, we illustrate the utility of the framework. This work offers developers actionable insights for selecting suitable technologies in LLM-based software system development, promoting robustness and scalability.