The Representational Status of Deep Learning Models
It addresses a foundational philosophical issue in AI for researchers and practitioners, but it is incremental as it builds on existing debates about representation.
This paper tackles the problem of clarifying the representational status of deep learning models, arguing that they represent targets in a relational sense but lack locally semantically decomposable representations, which has implications for explainable AI.
This paper aims to clarify the representational status of Deep Learning Models (DLMs). While commonly referred to as 'representations', what this entails is ambiguous due to a conflation of functional and relational conceptions of representation. This paper argues that while DLMs represent their targets in a relational sense, in general, we have no good reason to believe that DLMs encode locally semantically decomposable representations of their targets. That is, the representational capacity these models have is largely global, rather than decomposable into stable, local subrepresentations. This result has immediate implications for explainable AI (XAI) and directs attention toward exploring the global relational nature of deep learning representations and their relationship both to models more generally to understand their potential role in future scientific inquiry.