On Learning with LAD
This work addresses overfitting concerns for users of LAD classifiers, but it is incremental as it builds on existing LAD methods.
The paper tackled the problem of overfitting in classifiers from Logical Analysis of Data (LAD) by providing a theoretical justification based on estimating the VC dimension for DNF models with few cubic monomials, and confirmed this empirically.
The logical analysis of data, LAD, is a technique that yields two-class classifiers based on Boolean functions having disjunctive normal form (DNF) representation. Although LAD algorithms employ optimization techniques, the resulting binary classifiers or binary rules do not lead to overfitting. We propose a theoretical justification for the absence of overfitting by estimating the Vapnik-Chervonenkis dimension (VC dimension) for LAD models where hypothesis sets consist of DNFs with a small number of cubic monomials. We illustrate and confirm our observations empirically.