Aggregation Cross-Entropy for Sequence Recognition
This work addresses sequence recognition problems in fields like computer vision and natural language processing, presenting an incremental improvement with practical efficiency gains.
The paper tackles sequence recognition by proposing a novel loss function called aggregation cross-entropy (ACE), which achieves competitive performance to existing methods like CTC and attention mechanisms while offering faster implementation, inference, and lower storage requirements.
In this paper, we propose a novel method, aggregation cross-entropy (ACE), for sequence recognition from a brand new perspective. The ACE loss function exhibits competitive performance to CTC and the attention mechanism, with much quicker implementation (as it involves only four fundamental formulas), faster inference\back-propagation (approximately O(1) in parallel), less storage requirement (no parameter and negligible runtime memory), and convenient employment (by replacing CTC with ACE). Furthermore, the proposed ACE loss function exhibits two noteworthy properties: (1) it can be directly applied for 2D prediction by flattening the 2D prediction into 1D prediction as the input and (2) it requires only characters and their numbers in the sequence annotation for supervision, which allows it to advance beyond sequence recognition, e.g., counting problem. The code is publicly available at https://github.com/summerlvsong/Aggregation-Cross-Entropy.