Simpson's Paradox and the implications for medical trials
This highlights a fundamental limitation in medical trials, potentially affecting all researchers and practitioners who rely on such studies for decision-making.
The paper demonstrates that Simpson's Paradox can invalidate conclusions from randomized control trials by showing that no matter how many variables are considered, an unobserved confounding variable could theoretically reverse the results, such as indicating a drug is ineffective when it previously appeared effective.
This paper describes Simpson's paradox, and explains its serious implications for randomised control trials. In particular, we show that for any number of variables we can simulate the result of a controlled trial which uniformly points to one conclusion (such as 'drug is effective') for every possible combination of the variable states, but when a previously unobserved confounding variable is included every possible combination of the variables state points to the opposite conclusion ('drug is not effective'). In other words no matter how many variables are considered, and no matter how 'conclusive' the result, one cannot conclude the result is truly 'valid' since there is theoretically an unobserved confounding variable that could completely reverse the result.