CaTs and DAGs: Integrating Directed Acyclic Graphs with Transformers and Fully-Connected Neural Networks for Causally Constrained Predictions
This addresses the reliability issue for real-world applications of neural networks by integrating causal constraints, though it is incremental as it builds on existing neural network architectures.
The paper tackled the problem of neural networks lacking causal structure, which limits robustness and interpretability, by introducing Causal Fully-Connected Neural Networks (CFCNs) and Causal Transformers (CaTs) that operate under predefined Directed Acyclic Graph (DAG) constraints, resulting in improved robustness, reliability, and interpretability.
Artificial Neural Networks (ANNs), including fully-connected networks and transformers, are highly flexible and powerful function approximators, widely applied in fields like computer vision and natural language processing. However, their inability to inherently respect causal structures can limit their robustness, making them vulnerable to covariate shift and difficult to interpret/explain. This poses significant challenges for their reliability in real-world applications. In this paper, we introduce Causal Fully-Connected Neural Networks (CFCNs) and Causal Transformers (CaTs), two general model families designed to operate under predefined causal constraints, as specified by a Directed Acyclic Graph (DAG). These models retain the powerful function approximation abilities of traditional neural networks while adhering to the underlying structural constraints, improving robustness, reliability, and interpretability at inference time. This approach opens new avenues for deploying neural networks in more demanding, real-world scenarios where robustness and explainability is critical.