Semi-Periodic Activation for Time Series Classification
This addresses the problem of improving neural network performance for time series classification, though it appears incremental as it focuses on optimizing activation functions within existing frameworks.
The paper tackled the lack of research on activation functions for time series neural networks by proposing LeakySineLU, a new activation designed to maximize properties like boundedness and periodicity, and it achieved the best average ranking on 112 benchmark datasets for time series classification.
This paper investigates the lack of research on activation functions for neural network models in time series tasks. It highlights the need to identify essential properties of these activations to improve their effectiveness in specific domains. To this end, the study comprehensively analyzes properties, such as bounded, monotonic, nonlinearity, and periodicity, for activation in time series neural networks. We propose a new activation that maximizes the coverage of these properties, called LeakySineLU. We empirically evaluate the LeakySineLU against commonly used activations in the literature using 112 benchmark datasets for time series classification, obtaining the best average ranking in all comparative scenarios.