Aligning Robot and Human Representations
This addresses the problem of making robots more effective and acceptable for human users by ensuring their internal models reflect human values, though it is incremental as it builds on existing representation learning approaches.
The paper identifies representation misalignment as a problem in robotics, where learned robot representations fail to capture human concerns, and advocates for explicitly focusing on aligning these representations with humans in addition to learning tasks.
To act in the world, robots rely on a representation of salient task aspects: for example, to carry a coffee mug, a robot may consider movement efficiency or mug orientation in its behavior. However, if we want robots to act for and with people, their representations must not be just functional but also reflective of what humans care about, i.e. they must be aligned. We observe that current learning approaches suffer from representation misalignment, where the robot's learned representation does not capture the human's representation. We suggest that because humans are the ultimate evaluator of robot performance, we must explicitly focus our efforts on aligning learned representations with humans, in addition to learning the downstream task. We advocate that current representation learning approaches in robotics should be studied from the perspective of how well they accomplish the objective of representation alignment. We mathematically define the problem, identify its key desiderata, and situate current methods within this formalism. We conclude by suggesting future directions for exploring open challenges.