Toward a Modular Architecture for Embedded AI Agent Systems at the Edge
For researchers and engineers building autonomous AI systems on resource-constrained edge devices, this work offers a conceptual framework but lacks empirical validation, making it an incremental architectural proposal.
The paper proposes a modular reference architecture for deploying LLM-based agentic AI on embedded microcontrollers, addressing memory and energy constraints by decoupling on-device agents (compressed neural nets, rule-based logic) from cloud-augmented agents (small language models) with a governance layer for safety. No empirical results are provided; the contribution is architectural design principles and trade-off analysis.
The rise of Large Language Models (LLMs) has enabled agentic AI capable of complex reasoning and tool use; however, deploying such autonomy in pervasive computing environments remains challenging due to the strict memory and energy constraints of embedded microcontrollers. Existing frameworks typically assume server-class resources or continuous connectivity, leaving a gap for deeply embedded systems. This paper proposes a modular reference architecture for Embedded Agent Systems that bridges the divide between deterministic real-time control and agentic intelligence. We introduce a tiered design that decouples On-Device Agents - executing highly compressed neural networks and rule-based logic for low-latency, privacy-critical tasks - from Cloud-Augmented Agents that leverage Small Language Models (SLMs) for higher-level reasoning and planning. A key contribution is the integration of a cross-cutting Governance Layer, ensuring observability, policy enforcement, and safety across distributed fleets of autonomous devices. Rather than presenting purely empirical benchmarks, we analyze architectural design principles and trade-offs regarding latency, energy, and reliable execution in resource-constrained environments.