Generative multitask learning mitigates target-causing confounding
This addresses robustness issues in multitask learning for scenarios with unobserved confounders, though it is incremental as it modifies only the inference objective of existing methods.
The paper tackles the problem of target-causing confounders in multitask learning, which induce spurious dependencies between input and targets, and proposes generative multitask learning (GMTL) to mitigate this at inference time, resulting in improved robustness to target shift across four methods on Attributes of People and Taskonomy datasets.
We propose generative multitask learning (GMTL), a simple and scalable approach to causal representation learning for multitask learning. Our approach makes a minor change to the conventional multitask inference objective, and improves robustness to target shift. Since GMTL only modifies the inference objective, it can be used with existing multitask learning methods without requiring additional training. The improvement in robustness comes from mitigating unobserved confounders that cause the targets, but not the input. We refer to them as \emph{target-causing confounders}. These confounders induce spurious dependencies between the input and targets. This poses a problem for conventional multitask learning, due to its assumption that the targets are conditionally independent given the input. GMTL mitigates target-causing confounding at inference time, by removing the influence of the joint target distribution, and predicting all targets jointly. This removes the spurious dependencies between the input and targets, where the degree of removal is adjustable via a single hyperparameter. This flexibility is useful for managing the trade-off between in- and out-of-distribution generalization. Our results on the Attributes of People and Taskonomy datasets reflect an improved robustness to target shift across four multitask learning methods.