Extensional Properties of Recurrent Neural Networks
This result shows that verifying key functional properties of RNNs is fundamentally impossible in general, which is a foundational limitation for researchers and practitioners in machine learning and AI.
The paper tackles the problem of testing extensional properties in recurrent neural networks (RNNs), such as robustness or input clustering, and proves that any nontrivial extensional property is undecidable, analogous to Rice's theorem.
A property of a recurrent neural network (RNN) is called \emph{extensional} if, loosely speaking, it is a property of the function computed by the RNN rather than a property of the RNN algorithm. Many properties of interest in RNNs are extensional, for example, robustness against small changes of input or good clustering of inputs. Given an RNN, it is natural to ask whether it has such a property. We give a negative answer to the general question about testing extensional properties of RNNs. Namely, we prove a version of Rice's theorem for RNNs: any nontrivial extensional property of RNNs is undecidable.