PixVOD: Pixel-Distributed Direct Visual Odometry and Depth Estimation
This work addresses the inefficiency of transmitting raw pixel data off-sensor by enabling distributed computation on focal-plane sensor-processors, though it remains a proof-of-concept with no reported quantitative results.
PixVOD proposes a fully parallelizable visual odometry and depth estimation method that operates at the pixel level using Gaussian Belief Propagation, enabling on-sensor computation. It demonstrates feasibility on realistic datasets with a keyframe anchoring mechanism for geometric stability.
Images composed of 2D pixel arrays are the standard input to computer vision algorithms, yet many underlying computations can be distributed across pixels. Transmitting raw, redundant, and noisy pixel data off the sensor remains inefficient, motivating a shift toward focal-plane sensor-processors that perform a significant part of the computation directly within each pixel. We envision pixels synthesizing higher-level signals locally, reducing downstream load, and providing richer inputs for higher-level vision tasks. We propose a fully parallelizable form of visual odometry and depth estimation across pixels, where sensor-processors exchange information through Gaussian Belief Propagation (GBP) to achieve consensus about camera motion and infer depth from per-pixel photometric observations and a surface normal prior. To maintain geometric stability during optimization, we introduce a keyframe-like anchoring mechanism that regulates the effective baseline between frames, enabling consistent motion and depth updates. Our method is evaluated on realistic datasets, demonstrating the feasibility of GBP-based pixel-level distributed odometry and depth estimation with keyframe anchoring on-sensor. Project Page: https://www.shinjeongkim.com/pixvod/