X3D: Expanding Architectures for Efficient Video Recognition
This addresses the problem of high computational cost in video recognition for AI applications, offering a novel method for efficient model design.
The paper tackles efficient video recognition by introducing X3D, a family of networks that expand a 2D architecture along multiple axes to optimize accuracy-complexity trade-offs, achieving state-of-the-art performance with 4.8x fewer multiply-adds and 5.5x fewer parameters compared to previous work.
This paper presents X3D, a family of efficient video networks that progressively expand a tiny 2D image classification architecture along multiple network axes, in space, time, width and depth. Inspired by feature selection methods in machine learning, a simple stepwise network expansion approach is employed that expands a single axis in each step, such that good accuracy to complexity trade-off is achieved. To expand X3D to a specific target complexity, we perform progressive forward expansion followed by backward contraction. X3D achieves state-of-the-art performance while requiring 4.8x and 5.5x fewer multiply-adds and parameters for similar accuracy as previous work. Our most surprising finding is that networks with high spatiotemporal resolution can perform well, while being extremely light in terms of network width and parameters. We report competitive accuracy at unprecedented efficiency on video classification and detection benchmarks. Code will be available at: https://github.com/facebookresearch/SlowFast