Otimizacao de pesos e funcoes de ativacao de redes neurais aplicadas na previsao de series temporais
This work addresses the limited use of activation functions in neural networks for time series prediction, offering a novel optimization approach that could enhance forecasting accuracy for applications in fields like finance or weather prediction, though it appears incremental as it builds on existing methods like MLP and standard learning algorithms.
The authors tackled the problem of improving time series prediction by proposing a family of free-parameter asymmetric activation functions for neural networks, which satisfy the universal approximation theorem, and developed a methodology to simultaneously optimize these functions and network weights using simulated annealing, tabu search, and local learning, resulting in improved performance in adjustment and forecasting.
Neural Networks have been applied for time series prediction with good experimental results that indicate the high capacity to approximate functions with good precision. Most neural models used in these applications use activation functions with fixed parameters. However, it is known that the choice of activation function strongly influences the complexity and performance of the neural network and that a limited number of activation functions have been used. In this work, we propose the use of a family of free parameter asymmetric activation functions for neural networks and show that this family of defined activation functions satisfies the requirements of the universal approximation theorem. A methodology for the global optimization of this family of activation functions with free parameter and the weights of the connections between the processing units of the neural network is used. The central idea of the proposed methodology is to simultaneously optimize the weights and the activation function used in a multilayer perceptron network (MLP), through an approach that combines the advantages of simulated annealing, tabu search and a local learning algorithm, with the purpose of improving performance in the adjustment and forecasting of time series. We chose two learning algorithms: backpropagation with the term momentum (BPM) and LevenbergMarquardt (LM).