Efficient Per-Example Gradient Computations
arXiv:1510.01799v281 citations
Originality Synthesis-oriented
AI Analysis
This addresses a computational bottleneck for researchers and practitioners needing detailed gradient analysis, but it appears incremental as it focuses on efficiency improvements without claiming new applications.
The paper tackles the problem of efficiently computing the gradient norm of the loss function for each example in a neural network, achieving efficient per-example computations.
This technical report describes an efficient technique for computing the norm of the gradient of the loss function for a neural network with respect to its parameters. This gradient norm can be computed efficiently for every example.