SIFOTL: A Principled, Statistically-Informed Fidelity-Optimization Method for Tabular Learning
This addresses the problem of data shift analysis for healthcare decision-makers, offering an interpretable and privacy-conscious solution, though it appears incremental as it builds on existing methods like XGBoost and LLMs.
The paper tackled the challenge of identifying data shift factors in tabular datasets, particularly in healthcare, by proposing SIFOTL, a method that uses privacy-compliant summary statistics and achieves F1 scores up to 0.96, outperforming baselines like BigQuery Contribution Analysis (F1=0.46) and statistical tests (F1=0.20).
Identifying the factors driving data shifts in tabular datasets is a significant challenge for analysis and decision support systems, especially those focusing on healthcare. Privacy rules restrict data access, and noise from complex processes hinders analysis. To address this challenge, we propose SIFOTL (Statistically-Informed Fidelity-Optimization Method for Tabular Learning) that (i) extracts privacy-compliant data summary statistics, (ii) employs twin XGBoost models to disentangle intervention signals from noise with assistance from LLMs, and (iii) merges XGBoost outputs via a Pareto-weighted decision tree to identify interpretable segments responsible for the shift. Unlike existing analyses which may ignore noise or require full data access for LLM-based analysis, SIFOTL addresses both challenges using only privacy-safe summary statistics. Demonstrating its real-world efficacy, for a MEPS panel dataset mimicking a new Medicare drug subsidy, SIFOTL achieves an F1 score of 0.85, substantially outperforming BigQuery Contribution Analysis (F1=0.46) and statistical tests (F1=0.20) in identifying the segment receiving the subsidy. Furthermore, across 18 diverse EHR datasets generated based on Synthea ABM, SIFOTL sustains F1 scores of 0.86-0.96 without noise and >= 0.75 even with injected observational noise, whereas baseline average F1 scores range from 0.19-0.67 under the same tests. SIFOTL, therefore, provides an interpretable, privacy-conscious workflow that is empirically robust to observational noise.