CombU: A Combined Unit Activation for Fitting Mathematical Expressions with Neural Networks
This work addresses the challenge of improving neural network accuracy for mathematical expression fitting, though it appears incremental as it combines existing functions rather than introducing a new paradigm.
The paper tackles the problem of enhancing neural network performance by proposing CombU, a combined unit activation that uses different activation functions across dimensions and layers, and it outperforms six SOTA algorithms in 10 out of 16 metrics on mathematical expression datasets.
The activation functions are fundamental to neural networks as they introduce non-linearity into data relationships, thereby enabling deep networks to approximate complex data relations. Existing efforts to enhance neural network performance have predominantly focused on developing new mathematical functions. However, we find that a well-designed combination of existing activation functions within a neural network can also achieve this objective. In this paper, we introduce the Combined Units activation (CombU), which employs different activation functions at various dimensions across different layers. This approach can be theoretically proven to fit most mathematical expressions accurately. The experiments conducted on four mathematical expression datasets, compared against six State-Of-The-Art (SOTA) activation function algorithms, demonstrate that CombU outperforms all SOTA algorithms in 10 out of 16 metrics and ranks in the top three for the remaining six metrics.