ReLU Code Space: A Basis for Rating Network Quality Besides Accuracy
This work addresses the need for better metrics in neural network evaluation, particularly for safety and robustness, though it appears incremental as it builds on existing concepts of polyhedral bodies.
The authors tackled the problem of evaluating neural network quality beyond accuracy by proposing a ReLU activation code space with a truncated Hamming distance, which establishes an isometry to polyhedral bodies related to safety and robustness, and found that this space stores additional information on MNIST and CIFAR-10 datasets.
We propose a new metric space of ReLU activation codes equipped with a truncated Hamming distance which establishes an isometry between its elements and polyhedral bodies in the input space which have recently been shown to be strongly related to safety, robustness, and confidence. This isometry allows the efficient computation of adjacency relations between the polyhedral bodies. Experiments on MNIST and CIFAR-10 indicate that information besides accuracy might be stored in the code space.