Divide and Conquer Local Average Regression
This work addresses scalability issues in regression analysis for large datasets, but it is incremental as it builds on existing divide-and-conquer and local average regression methods.
The paper tackles the problem of applying divide-and-conquer strategies to local average regression for massive datasets, finding that the basic approach has strong restrictions on the number of data blocks. It proposes two variants that achieve optimal learning rates with milder or no such restrictions, as verified by extensive experiments.
The divide and conquer strategy, which breaks a massive data set into a se- ries of manageable data blocks, and then combines the independent results of data blocks to obtain a final decision, has been recognized as a state-of-the-art method to overcome challenges of massive data analysis. In this paper, we merge the divide and conquer strategy with local average regression methods to infer the regressive relationship of input-output pairs from a massive data set. After theoretically analyzing the pros and cons, we find that although the divide and conquer local average regression can reach the optimal learning rate, the restric- tion to the number of data blocks is a bit strong, which makes it only feasible for small number of data blocks. We then propose two variants to lessen (or remove) this restriction. Our results show that these variants can achieve the optimal learning rate with much milder restriction (or without such restriction). Extensive experimental studies are carried out to verify our theoretical assertions.