Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Network
This addresses segmentation challenges in robotic grasping under real-world dynamic conditions, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackles panoptic segmentation for robotic grasping under dynamic conditions like occlusion and motion blur by proposing a Graph Mixer Neural Network with a collaborative contextual mixing layer on 3D event graphs. The method outperforms state-of-the-art approaches on the ESD Dataset, achieving higher mean intersection over union and pixel accuracy.
In the context of robotic grasping, object segmentation encounters several difficulties when faced with dynamic conditions such as real-time operation, occlusion, low lighting, motion blur, and object size variability. In response to these challenges, we propose the Graph Mixer Neural Network that includes a novel collaborative contextual mixing layer, applied to 3D event graphs formed on asynchronous events. The proposed layer is designed to spread spatiotemporal correlation within an event graph at four nearest neighbor levels parallelly. We evaluate the effectiveness of our proposed method on the Event-based Segmentation (ESD) Dataset, which includes five unique image degradation challenges, including occlusion, blur, brightness, trajectory, scale variance, and segmentation of known and unknown objects. The results show that our proposed approach outperforms state-of-the-art methods in terms of mean intersection over the union and pixel accuracy. Code available at: https://github.com/sanket0707/GNN-Mixer.git