Relation-First Modeling Paradigm for Causal Representation Learning toward the Development of AGI
This foundational work addresses causal representation learning challenges for advancing toward AGI, though it appears incremental as it builds on existing causal modeling efforts.
The paper tackles the limitations of traditional i.i.d.-based learning in capturing causal relationships by proposing a new 'relation-first' modeling paradigm, with experiments validating the efficacy of its practical implementation, Relation-Indexed Representation Learning (RIRL).
The traditional i.i.d.-based learning paradigm faces inherent challenges in addressing causal relationships, which has become increasingly evident with the rise of applications in causal representation learning. Our understanding of causality naturally requires a perspective as the creator rather than observer, as the ``what...if'' questions only hold within the possible world we conceive. The traditional perspective limits capturing dynamic causal outcomes and leads to compensatory efforts such as the reliance on hidden confounders. This paper lays the groundwork for the new perspective, which enables the \emph{relation-first} modeling paradigm for causality. Also, it introduces the Relation-Indexed Representation Learning (RIRL) as a practical implementation, supported by experiments that validate its efficacy.