Learning Independent Causal Mechanisms
This work addresses the challenge of transfer learning by enabling the extraction of autonomous generative modules that can be reused across different problems, though it appears incremental in its approach.
The paper tackles the problem of recovering independent causal mechanisms from transformed data using an unsupervised algorithm with competing experts, and demonstrates that the learned mechanisms generalize to novel domains in image experiments.
Statistical learning relies upon data sampled from a distribution, and we usually do not care what actually generated it in the first place. From the point of view of causal modeling, the structure of each distribution is induced by physical mechanisms that give rise to dependences between observables. Mechanisms, however, can be meaningful autonomous modules of generative models that make sense beyond a particular entailed data distribution, lending themselves to transfer between problems. We develop an algorithm to recover a set of independent (inverse) mechanisms from a set of transformed data points. The approach is unsupervised and based on a set of experts that compete for data generated by the mechanisms, driving specialization. We analyze the proposed method in a series of experiments on image data. Each expert learns to map a subset of the transformed data back to a reference distribution. The learned mechanisms generalize to novel domains. We discuss implications for transfer learning and links to recent trends in generative modeling.