On The Impact of Merge Request Deviations on Code Review Practices
This work addresses a specific issue for software engineering practitioners by providing tools to optimize code review efforts and improve reliability in ML-based analytics, though it is incremental in nature.
The paper tackled the problem of Merge Request (MR) deviations, such as drafts or rebases, which occur in 37.02% of MRs and bias code review analytics, by proposing a detection method with 91% accuracy and showing that excluding deviations improves ML model performance for predicting review completion time in 53.33% of cases, with up to 2.25x gains.
Code review is a key practice in software engineering, ensuring quality and collaboration. However, industrial Merge Request (MR) workflows often deviate from standardized review processes, with many MRs serving non-review purposes (e.g., drafts, rebases, or dependency updates). We term these cases deviations and hypothesize that ignoring them biases analytics and undermines ML models for review analysis. We identify seven deviation categories, occurring in 37.02% of MRs, and propose a few-shot learning detection method (91% accuracy). By excluding deviations, ML models predicting review completion time improve performance in 53.33% of cases (up to 2.25x) and exhibit significant shifts in feature importance (47% overall, 60% top-*k*). Our contributions include: (1) a taxonomy of MR deviations, (2) an AI-driven detection approach, and (3) empirical evidence of their impact on ML-based review analytics. This work aids practitioners in optimizing review efforts and ensuring reliable insights.