Two ways to knowledge?
This highlights a fundamental tension between AI-driven knowledge acquisition and traditional scientific explainability, which is crucial for researchers in AI and physics.
The study found that transformer weight matrices in physical applications appear random and lack direct correspondence to the underlying physical structures, suggesting machine learning and the scientific method may be complementary paths to knowledge, though explainability remains elusive.
It is shown that the weight matrices of transformer-based machine learning applications to the solution of two representative physical applications show a random-like character which bears no directly recognizable link to the physical and mathematical structure of the physical problem under study. This suggests that machine learning and the scientific method may represent two distinct and potentially complementary paths to knowledge, even though a strict notion of explainability in terms of direct correspondence between network parameters and physical structures may remain out of reach. It is also observed that drawing a parallel between transformer operation and (generalized) path-integration techniques may account for the random-like nature of the weights, but still does not resolve the tension with explainability. We conclude with some general comments on the hazards of gleaning knowledge without the benefit of Insight.