Robust Point Cloud Reinforcement Learning via PCA-Based Canonicalization
This addresses reliability issues in robotic control by reducing viewpoint-induced inconsistencies, though it is incremental as it builds on existing point cloud methods.
The paper tackles the problem of point cloud reinforcement learning being sensitive to camera pose mismatches, proposing a PCA-based canonicalization framework that improves robustness to unseen camera poses in robotic tasks.
Reinforcement Learning (RL) from raw visual input has achieved impressive successes in recent years, yet it remains fragile to out-of-distribution variations such as changes in lighting, color, and viewpoint. Point Cloud Reinforcement Learning (PC-RL) offers a promising alternative by mitigating appearance-based brittleness, but its sensitivity to camera pose mismatches continues to undermine reliability in realistic settings. To address this challenge, we propose PCA Point Cloud (PPC), a canonicalization framework specifically tailored for downstream robotic control. PPC maps point clouds under arbitrary rigid-body transformations to a unique canonical pose, aligning observations to a consistent frame, thereby substantially decreasing viewpoint-induced inconsistencies. In our experiments, we show that PPC improves robustness to unseen camera poses across challenging robotic tasks, providing a principled alternative to domain randomization.