Network Inversion of Binarised Neural Nets
This work addresses interpretability and trustworthiness in safety-critical applications for resource-constrained environments, representing an incremental advancement in network inversion techniques.
The paper tackles the problem of interpreting binarised neural networks (BNNs) by developing a network inversion method that reconstructs inputs from internal representations, enabling both inference and inversion through encoding into CNF formulas.
While the deployment of neural networks, yielding impressive results, becomes more prevalent in various applications, their interpretability and understanding remain a critical challenge. Network inversion, a technique that aims to reconstruct the input space from the model's learned internal representations, plays a pivotal role in unraveling the black-box nature of input to output mappings in neural networks. In safety-critical scenarios, where model outputs may influence pivotal decisions, the integrity of the corresponding input space is paramount, necessitating the elimination of any extraneous "garbage" to ensure the trustworthiness of the network. Binarised Neural Networks (BNNs), characterized by binary weights and activations, offer computational efficiency and reduced memory requirements, making them suitable for resource-constrained environments. This paper introduces a novel approach to invert a trained BNN by encoding it into a CNF formula that captures the network's structure, allowing for both inference and inversion.