Consensus Feature Network for Scene Parsing
This work addresses pixel-level inconsistency in scene parsing for computer vision applications, representing an incremental improvement.
The paper tackles inconsistent pixel-level predictions in scene parsing by proposing two lightweight transform units, Instance Consensus Transform (ICT) and Category Consensus Transform (CCT), to learn consensus features; the resulting Consensus Feature Network (CFNet) achieves competitive performance on four datasets including Cityscapes and COCO Stuff.
Scene parsing is challenging as it aims to assign one of the semantic categories to each pixel in scene images. Thus, pixel-level features are desired for scene parsing. However, classification networks are dominated by the discriminative portion, so directly applying classification networks to scene parsing will result in inconsistent parsing predictions within one instance and among instances of the same category. To address this problem, we propose two transform units to learn pixel-level consensus features. One is an Instance Consensus Transform (ICT) unit to learn the instance-level consensus features by aggregating features within the same instance. The other is a Category Consensus Transform (CCT) unit to pursue category-level consensus features through keeping the consensus of features among instances of the same category in scene images. The proposed ICT and CCT units are lightweight, data-driven and end-to-end trainable. The features learned by the two units are more coherent in both instance-level and category-level. Furthermore, we present the Consensus Feature Network (CFNet) based on the proposed ICT and CCT units, and demonstrate the effectiveness of each component in our method by performing extensive ablation experiments. Finally, our proposed CFNet achieves competitive performance on four datasets, including Cityscapes, Pascal Context, CamVid, and COCO Stuff.