Act, Think or Abstain: Complexity-Aware Adaptive Inference for Vision-Language-Action Models
This work addresses the problem of inefficient resource allocation and lack of uncertainty estimation in VLA models by introducing a complexity-aware adaptive inference framework, which is an incremental improvement for robotics and embodied AI.
This paper proposes an adaptive framework for Vision-Language-Action (VLA) models that dynamically routes execution based on perceived state complexity. The system can immediately execute known tasks, reason about ambiguous scenarios, or halt execution for anomalies. The vision-only configuration achieves an 80% F1-Score for task complexity detection using only 5% of training data.
Current research on Vision-Language-Action (VLA) models predominantly focuses on enhancing generalization through established reasoning techniques. While effective, these improvements invariably increase computational complexity and inference latency. Furthermore, these mechanisms are typically applied indiscriminately, resulting in the inefficient allocation of resources for trivial tasks while simultaneously failing to provide the uncertainty estimation necessary to prevent catastrophic failure on out-of-distribution tasks. Inspired by human cognition, we propose an adaptive framework that dynamically routes VLA execution based on the complexity of the perceived state. Our approach transforms the VLA's vision-language backbone into an active detection tool by projecting latent embeddings into an ensemble of parametric and non-parametric estimators. This allows the system to execute known tasks immediately (Act), reason about ambiguous scenarios (Think), and preemptively halt execution when encountering significant physical or semantic anomalies (Abstain). In our empirical analysis, we observe a phenomenon where visual embeddings alone are superior for inferring task complexity due to the semantic invariance of language. Evaluated on the LIBERO and LIBERO-PRO benchmarks as well as on a real robot, our vision-only configuration achieves 80% F1-Score using as little as 5% of training data, establishing itself as a reliable and efficient task complexity detector.