Score-based Causal Representation Learning with Interventions
This work addresses causal representation learning, a foundational challenge in machine learning for understanding underlying causal mechanisms from data, with potential applications in domains like healthcare or economics, though it appears incremental as it builds on existing intervention-based methods.
This paper tackles the problem of recovering latent causal variables and their directed acyclic graph from observed data under unknown linear transformations, using interventions, and shows that perfect recovery of the DAG structure is possible with soft interventions, and unique recovery up to scaling and ordering with hard interventions.
This paper studies the causal representation learning problem when the latent causal variables are observed indirectly through an unknown linear transformation. The objectives are: (i) recovering the unknown linear transformation (up to scaling) and (ii) determining the directed acyclic graph (DAG) underlying the latent variables. Sufficient conditions for DAG recovery are established, and it is shown that a large class of non-linear models in the latent space (e.g., causal mechanisms parameterized by two-layer neural networks) satisfy these conditions. These sufficient conditions ensure that the effect of an intervention can be detected correctly from changes in the score. Capitalizing on this property, recovering a valid transformation is facilitated by the following key property: any valid transformation renders latent variables' score function to necessarily have the minimal variations across different interventional environments. This property is leveraged for perfect recovery of the latent DAG structure using only \emph{soft} interventions. For the special case of stochastic \emph{hard} interventions, with an additional hypothesis testing step, one can also uniquely recover the linear transformation up to scaling and a valid causal ordering.