Balancing Information Accuracy and Response Timeliness in Networked LLMs
This work addresses practical deployment issues for LLMs in applications like scientific discovery and content generation, though it is incremental in optimizing existing aggregation methods.
The paper tackles the challenge of balancing accuracy and timeliness in networked LLM systems by aggregating outputs from specialized models, showing that aggregated responses achieve higher accuracy than individual LLMs, especially when models have similar standalone performance.
Recent advancements in Large Language Models (LLMs) have transformed many fields including scientific discovery, content generation, biomedical text mining, and educational technology. However, the substantial requirements for training data, computational resources, and energy consumption pose significant challenges for their practical deployment. A promising alternative is to leverage smaller, specialized language models and aggregate their outputs to improve overall response quality. In this work, we investigate a networked LLM system composed of multiple users, a central task processor, and clusters of topic-specialized LLMs. Each user submits categorical binary (true/false) queries, which are routed by the task processor to a selected cluster of $m$ LLMs. After gathering individual responses, the processor returns a final aggregated answer to the user. We characterize both the information accuracy and response timeliness in this setting, and formulate a joint optimization problem to balance these two competing objectives. Our extensive simulations demonstrate that the aggregated responses consistently achieve higher accuracy than those of individual LLMs. Notably, this improvement is more significant when the participating LLMs exhibit similar standalone performance.