Real-World Robot Applications of Foundation Models: A Review
It addresses the integration of advanced AI models into robotics for researchers and practitioners, but is incremental as it summarizes existing developments rather than introducing new methods.
This paper reviews the practical application of foundation models, such as LLMs and VLMs, in real-world robotics, focusing on how they replace specific components in existing robot systems to enhance perception, motion planning, and control.
Recent developments in foundation models, like Large Language Models (LLMs) and Vision-Language Models (VLMs), trained on extensive data, facilitate flexible application across different tasks and modalities. Their impact spans various fields, including healthcare, education, and robotics. This paper provides an overview of the practical application of foundation models in real-world robotics, with a primary emphasis on the replacement of specific components within existing robot systems. The summary encompasses the perspective of input-output relationships in foundation models, as well as their role in perception, motion planning, and control within the field of robotics. This paper concludes with a discussion of future challenges and implications for practical robot applications.