Imputation-free Learning of Tabular Data with Missing Values using Incremental Feature Partitions in Transformer
This addresses data quality and reliability concerns for researchers and practitioners working with incomplete tabular data, offering a novel deep learning approach that is robust to varying missing value types and rates.
The paper tackles the problem of learning from tabular data with missing values by proposing an imputation-free incremental attention learning (IFIAL) method, which avoids synthetic imputation and achieves superior classification performance, ranking first on average across 17 diverse datasets compared to 11 state-of-the-art methods.
Tabular data sets with varying missing values are prepared for machine learning using an arbitrary imputation strategy. Synthetic values generated by imputation models often raise concerns about data quality and the reliability of data-driven outcomes. To address these concerns, this article proposes an imputation-free incremental attention learning (IFIAL) method for tabular data. A pair of attention masks is derived and retrofitted to a transformer to directly streamline tabular data without imputing or initializing missing values. The proposed method incrementally learns partitions of overlapping and fixed-size feature sets to enhance the efficiency and performance of the transformer. The average classification performance rank order across 17 diverse tabular data sets highlights the superiority of IFIAL over 11 state-of-the-art learning methods with or without missing value imputations. Further experiments substantiate the robustness of IFIAL against varying missing value types and rates compared to methods involving missing value imputation. Our analysis reveals that a feature partition size of half the original feature space is, both computationally and in terms of accuracy, the best choice for the proposed incremental learning. The proposed method is one of the first solutions to enable deep attention learning of tabular data without requiring missing-value imputation. The source code for this paper is publicly available.