Position: Avoid Overstretching LLMs for every Enterprise Task
For enterprise AI practitioners, this position paper challenges the prevailing trend of deploying LLMs for all tasks, advocating for a more efficient and reliable modular approach.
The paper argues that using large language models as monolithic engines for enterprise tasks is inefficient and unreliable, proposing instead a modular architecture where LLMs serve as interfaces for structured extraction while computation and knowledge are externalized to dedicated components. Theoretical evidence shows finite-capacity models cannot fully capture enterprise knowledge, and formal proofs demonstrate modular systems are more reliable and maintainable.
Enterprise workloads are dominated by deterministic, structured, and knowledge-dependent tasks operating under strict cost, latency, and reliability constraints. While these are often addressed through large language model (LLM) deployment or distillation into smaller models, we argue this is inefficient, unreliable, and misaligned with enterprise task structures. Instead, AI systems should treat language models as interfaces rather than monolithic engines, externalizing knowledge and computation into dedicated components for greater reliability, scalability, and transparency. Our theoretical evidences show that finite-capacity models cannot fully capture the breadth of knowledge required for enterprise tasks, creating inherent limits to efficiency and interpretability. Building on this, we take the position that language models should primarily be used for structured extraction in deterministic enterprise workflows, while computation and storage are delegated to knowledge bases and symbolic procedures. We formally demonstrate that such modular architectures are more reliable and maintainable than monolithic frameworks, offering a sustainable foundation for enterprise tasks.