Interactive Simulations of Backdoors in Neural Networks
This work addresses security vulnerabilities in deployed AI systems, though it is incremental as it builds on existing cryptographic and defense techniques.
The paper tackles the problem of understanding and defending against cryptographic backdoors in AI models by developing a web-based simulation playground that allows users to plant, activate, and defend backdoors in neural networks, with simulations available online.
This work addresses the problem of planting and defending cryptographic-based backdoors in artificial intelligence (AI) models. The motivation comes from our lack of understanding and the implications of using cryptographic techniques for planting undetectable backdoors under theoretical assumptions in the large AI model systems deployed in practice. Our approach is based on designing a web-based simulation playground that enables planting, activating, and defending cryptographic backdoors in neural networks (NN). Simulations of planting and activating backdoors are enabled for two scenarios: in the extension of NN model architecture to support digital signature verification and in the modified architectural block for non-linear operators. Simulations of backdoor defense against backdoors are available based on proximity analysis and provide a playground for a game of planting and defending against backdoors. The simulations are available at https://pages.nist.gov/nn-calculator