Regression with Linear Factored Functions
This addresses the curse of dimensionality for applications using empirically estimated functions, such as belief propagation and reinforcement learning, offering a method to break computational bottlenecks, though it appears incremental in its specific approach.
The paper tackles the curse of dimensionality in regression by introducing a novel algorithm that learns linear factored functions (LFF), which enable analytical solutions to integrals and point-wise products, speeding up computations in applications like belief propagation and reinforcement learning. The algorithm performs competitively with Gaussian processes on benchmarks, achieving compact representations with 4-9 factored basis functions on average.
Many applications that use empirically estimated functions face a curse of dimensionality, because the integrals over most function classes must be approximated by sampling. This paper introduces a novel regression-algorithm that learns linear factored functions (LFF). This class of functions has structural properties that allow to analytically solve certain integrals and to calculate point-wise products. Applications like belief propagation and reinforcement learning can exploit these properties to break the curse and speed up computation. We derive a regularized greedy optimization scheme, that learns factored basis functions during training. The novel regression algorithm performs competitively to Gaussian processes on benchmark tasks, and the learned LFF functions are with 4-9 factored basis functions on average very compact.