Efficient Stepwise Selection in Decomposable Models
This work addresses computational efficiency in model selection for statisticians and data scientists, but it is incremental as it builds on existing decomposable model frameworks.
The paper tackles the problem of efficiently performing stepwise selection in decomposable models by characterizing edges that can be added while maintaining decomposability and providing an O(1) time per edge enumeration algorithm, with analysis of the complete procedure's complexity for minimizing Kullback-Leibler distance.
In this paper, we present an efficient way of performing stepwise selection in the class of decomposable models. The main contribution of the paper is a simple characterization of the edges that canbe added to a decomposable model while keeping the resulting model decomposable and an efficient algorithm for enumerating all such edges for a given model in essentially O(1) time per edge. We also discuss how backward selection can be performed efficiently using our data structures.We also analyze the complexity of the complete stepwise selection procedure, including the complexity of choosing which of the eligible dges to add to (or delete from) the current model, with the aim ofminimizing the Kullback-Leibler distance of the resulting model from the saturated model for the data.