Beyond Softmax: Dual-Branch Sigmoid Architecture for Accurate Class Activation Maps
This addresses a fundamental limitation in visualization tools for deep networks, offering a simple, architecture-agnostic solution for researchers and practitioners in computer vision, though it is incremental as it builds on existing CAM variants.
The paper tackles distortions in Class Activation Mapping (CAM) methods caused by softmax classifiers by proposing a dual-branch sigmoid architecture that decouples localization from classification, resulting in improved explanation fidelity and consistent Top-1 Localization gains without accuracy drops on fine-grained tasks and WSOL benchmarks.
Class Activation Mapping (CAM) and its extensions have become indispensable tools for visualizing the evidence behind deep network predictions. However, by relying on a final softmax classifier, these methods suffer from two fundamental distortions: additive logit shifts that arbitrarily bias importance scores, and sign collapse that conflates excitatory and inhibitory features. We propose a simple, architecture-agnostic dual-branch sigmoid head that decouples localization from classification. Given any pretrained model, we clone its classification head into a parallel branch ending in per-class sigmoid outputs, freeze the original softmax head, and fine-tune only the sigmoid branch with class-balanced binary supervision. At inference, softmax retains recognition accuracy, while class evidence maps are generated from the sigmoid branch -- preserving both magnitude and sign of feature contributions. Our method integrates seamlessly with most CAM variants and incurs negligible overhead. Extensive evaluations on fine-grained tasks (CUB-200-2011, Stanford Cars) and WSOL benchmarks (ImageNet-1K, OpenImages30K) show improved explanation fidelity and consistent Top-1 Localization gains -- without any drop in classification accuracy. Code is available at https://github.com/finallyupper/beyond-softmax.