Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge
This work addresses regression tasks in machine learning by incorporating causal knowledge, offering a novel method for improving predictions, though it appears incremental as it builds on existing DAG frameworks.
The paper tackled the problem of discarding causal knowledge in regression tasks by showing that collider structures in DAGs provide inductive biases that improve predictive performance. The result included a proven generalization benefit and performance gains in experiments on synthetic and climate data.
A directed acyclic graph (DAG) provides valuable prior knowledge that is often discarded in regression tasks in machine learning. We show that the independences arising from the presence of collider structures in DAGs provide meaningful inductive biases, which constrain the regression hypothesis space and improve predictive performance. We introduce collider regression, a framework to incorporate probabilistic causal knowledge from a collider in a regression problem. When the hypothesis space is a reproducing kernel Hilbert space, we prove a strictly positive generalisation benefit under mild assumptions and provide closed-form estimators of the empirical risk minimiser. Experiments on synthetic and climate model data demonstrate performance gains of the proposed methodology.