Power Consumption Variation over Activation Functions
This addresses power efficiency for machine learning practitioners, but it is incremental as it focuses on a specific architectural factor.
The paper tackled the problem of power consumption variation in machine learning models by estimating power usage across different activation functions, finding substantial differences that can inform reductions in hardware performance.
The power that machine learning models consume when making predictions can be affected by a model's architecture. This paper presents various estimates of power consumption for a range of different activation functions, a core factor in neural network model architecture design. Substantial differences in hardware performance exist between activation functions. This difference informs how power consumption in machine learning models can be reduced.