The Impact of Sampling and Rule Set Size on Generated Fuzzy Inference System Predictive Accuracy: Analysis of a Software Engineering Data Set
This study provides insights into the sensitivity of fuzzy inference system performance to model-building decisions for software engineering practitioners using predictive modeling.
This paper investigates the impact of sampling and rule set size on the predictive accuracy of fuzzy inference systems (FIS) applied to software engineering data. They found no consistent pattern between rule set size and predictive performance, but a rule set compiled from multiple samples generally yielded more accurate predictions than single-sample rule sets.
Software project management makes extensive use of predictive modeling to estimate product size, defect proneness and development effort. Although uncertainty is acknowledged in these tasks, fuzzy inference systems, designed to cope well with uncertainty, have received only limited attention in the software engineering domain. In this study we empirically investigate the impact of two choices on the predictive accuracy of generated fuzzy inference systems when applied to a software engineering data set: sampling of observations for training and testing; and the size of the rule set generated using fuzzy c-means clustering. Over ten samples we found no consistent pattern of predictive performance given certain rule set size. We did find, however, that a rule set compiled from multiple samples generally resulted in more accurate predictions than single sample rule sets. More generally, the results provide further evidence of the sensitivity of empirical analysis outcomes to specific model-building decisions.