Principles from Clinical Research for NLP Model Generalization
This work addresses the issue of unreliable generalization evaluation in NLP for researchers and practitioners, offering a framework to enhance model reliability, though it is incremental by adapting existing clinical concepts.
The paper tackles the problem of assessing generalization in NLP models by drawing principles from clinical research, emphasizing the need for internal validity to avoid spurious correlations and improve model robustness, particularly in relation extraction tasks and generative large language models.
The NLP community typically relies on performance of a model on a held-out test set to assess generalization. Performance drops observed in datasets outside of official test sets are generally attributed to "out-of-distribution" effects. Here, we explore the foundations of generalizability and study the factors that affect it, articulating lessons from clinical studies. In clinical research, generalizability is an act of reasoning that depends on (a) internal validity of experiments to ensure controlled measurement of cause and effect, and (b) external validity or transportability of the results to the wider population. We demonstrate how learning spurious correlations, such as the distance between entities in relation extraction tasks, can affect a model's internal validity and in turn adversely impact generalization. We, therefore, present the need to ensure internal validity when building machine learning models in NLP. Our recommendations also apply to generative large language models, as they are known to be sensitive to even minor semantic preserving alterations. We also propose adapting the idea of matching in randomized controlled trials and observational studies to NLP evaluation to measure causation.