Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning
This work addresses the integration of statistical and algorithmic methods for statisticians and machine learning practitioners, but it is incremental as it builds on existing debates and literature.
The paper examines the evolution of the divide between data modeling and algorithmic modeling in statistics, highlighting how recent statistical developments in Random Forest methods have blended these approaches, while also noting the limitations of a prediction-first philosophy.
In 2001, Leo Breiman wrote of a divide between "data modeling" and "algorithmic modeling" cultures. Twenty years later this division feels far more ephemeral, both in terms of assigning individuals to camps, and in terms of intellectual boundaries. We argue that this is largely due to the "data modelers" incorporating algorithmic methods into their toolbox, particularly driven by recent developments in the statistical understanding of Breiman's own Random Forest methods. While this can be simplistically described as "Breiman won", these same developments also expose the limitations of the prediction-first philosophy that he espoused, making careful statistical analysis all the more important. This paper outlines these exciting recent developments in the random forest literature which, in our view, occurred as a result of a necessary blending of the two ways of thinking Breiman originally described. We also ask what areas statistics and statisticians might currently overlook.