POTTER: Pooling Attention Transformer for Efficient Human Mesh Recovery
This work addresses the need for efficient human mesh recovery models for real-world applications, offering a significant reduction in resource usage compared to existing methods.
The paper tackles the problem of high memory and computational costs in transformer-based human mesh recovery from monocular images by proposing POTTER, a lightweight transformer architecture that reduces parameters to 7% and operations to 14% of the SOTA while maintaining performance on Human3.6M and 3DPW datasets.
Transformer architectures have achieved SOTA performance on the human mesh recovery (HMR) from monocular images. However, the performance gain has come at the cost of substantial memory and computational overhead. A lightweight and efficient model to reconstruct accurate human mesh is needed for real-world applications. In this paper, we propose a pure transformer architecture named POoling aTtention TransformER (POTTER) for the HMR task from single images. Observing that the conventional attention module is memory and computationally expensive, we propose an efficient pooling attention module, which significantly reduces the memory and computational cost without sacrificing performance. Furthermore, we design a new transformer architecture by integrating a High-Resolution (HR) stream for the HMR task. The high-resolution local and global features from the HR stream can be utilized for recovering more accurate human mesh. Our POTTER outperforms the SOTA method METRO by only requiring 7% of total parameters and 14% of the Multiply-Accumulate Operations on the Human3.6M (PA-MPJPE metric) and 3DPW (all three metrics) datasets. The project webpage is https://zczcwh.github.io/potter_page.