Beyond Planar Symmetry: Modeling human perception of reflection and rotation symmetries in the wild
This work addresses the challenge of capturing human-like symmetry perception for computer vision applications, representing an incremental advance in symmetry detection models.
The authors tackled the problem of modeling human perception of reflection and rotation symmetries in real-world images by developing Sym-NET, a deep-learning neural network trained on human-labeled data from MS-COCO, which significantly outperformed existing computer vision algorithms on standard testsets and unseen photos.
Humans take advantage of real world symmetries for various tasks, yet capturing their superb symmetry perception mechanism with a computational model remains elusive. Motivated by a new study demonstrating the extremely high inter-person accuracy of human perceived symmetries in the wild, we have constructed the first deep-learning neural network for reflection and rotation symmetry detection (Sym-NET), trained on photos from MS-COCO (Microsoft-Common Object in COntext) dataset with nearly 11K consistent symmetry-labels from more than 400 human observers. We employ novel methods to convert discrete human labels into symmetry heatmaps, capture symmetry densely in an image and quantitatively evaluate Sym-NET against multiple existing computer vision algorithms. On CVPR 2013 symmetry competition testsets and unseen MS-COCO photos, Sym-NET significantly outperforms all other competitors. Beyond mathematically well-defined symmetries on a plane, Sym-NET demonstrates abilities to identify viewpoint-varied 3D symmetries, partially occluded symmetrical objects, and symmetries at a semantic level.