An End-to-End Robot Architecture to Manipulate Non-Physical State Changes of Objects
This addresses the challenge of non-physical state manipulation for humanoid robots in tasks like smart device operation, though it is incremental as it builds on existing task learning methods.
The paper tackles the problem of enabling humanoid robots to manipulate objects without physical movements, such as playing mobile games, by introducing an integrated end-to-end architecture that connects sensors to actuators, and demonstrates that the robot can play the 2048 game as accurately as in a simulated environment.
With the advance in robotic hardware and intelligent software, humanoid robot plays an important role in various tasks including service for human assistance and heavy job for hazardous industry. Recent advances in task learning enable humanoid robots to conduct dexterous manipulation tasks such as grasping objects and assembling parts of furniture. Operating objects without physical movements is an even more challenging task for humanoid robot because effects of actions may not be clearly seen in the physical configuration space and meaningful actions could be very complex in a long time horizon. As an example, playing a mobile game in a smart device has such challenges because both swipe actions and complex state transitions inside the smart devices in a long time horizon. In this paper, we solve this problem by introducing an integrated architecture which connects end-to-end dataflow from sensors to actuators in a humanoid robot to operate smart devices. We implement our integrated architecture in the Baxter Research Robot and experimentally demonstrate that the robot with our architecture could play a challenging mobile game, the 2048 game, as accurate as in a simulated environment.