TwistNet-2D: Learning Second-Order Channel Interactions via Spiral Twisting for Texture Recognition
This work addresses texture and fine-grained recognition by proposing a novel method to encode local channel interactions, offering a lightweight solution that outperforms larger models, though it is incremental in nature.
The paper tackled the problem of capturing second-order channel interactions for texture recognition by introducing TwistNet-2D, a lightweight module that computes local pairwise channel products with spatial displacement, resulting in consistent performance improvements over various baselines across four benchmarks with only 3.5% additional parameters and 2% additional FLOPs.
Second-order feature statistics are central to texture recognition, yet current methods face a fundamental tension: bilinear pooling and Gram matrices capture global channel correlations but collapse spatial structure, while self-attention models spatial context through weighted aggregation rather than explicit pairwise feature interactions. We introduce TwistNet-2D, a lightweight module that computes \emph{local} pairwise channel products under directional spatial displacement, jointly encoding where features co-occur and how they interact. The core component, Spiral-Twisted Channel Interaction (STCI), shifts one feature map along a prescribed direction before element-wise channel multiplication, thereby capturing the cross-position co-occurrence patterns characteristic of structured and periodic textures. Aggregating four directional heads with learned channel reweighting and injecting the result through a sigmoid-gated residual path, \TwistNet incurs only 3.5% additional parameters and 2% additional FLOPs over ResNet-18, yet consistently surpasses both parameter-matched and substantially larger baselines -- including ConvNeXt, Swin Transformer, and hybrid CNN--Transformer architectures -- across four texture and fine-grained recognition benchmarks.