Efficient Representation of the Activation Space in Deep Neural Networks
This addresses memory and privacy concerns for researchers and practitioners using DNN activations in tasks like anomaly detection and adversarial attack detection, though it is incremental as it builds on existing p-value methods.
The paper tackles the problem of efficiently representing activation spaces in deep neural networks by proposing a model-agnostic framework using node-specific histograms to compute p-values without storing raw data, resulting in a 30% reduction in memory usage and up to 4 times faster computation while maintaining state-of-the-art detection performance.
The representations of the activation space of deep neural networks (DNNs) are widely utilized for tasks like natural language processing, anomaly detection and speech recognition. Due to the diverse nature of these tasks and the large size of DNNs, an efficient and task-independent representation of activations becomes crucial. Empirical p-values have been used to quantify the relative strength of an observed node activation compared to activations created by already-known inputs. Nonetheless, keeping raw data for these calculations increases memory resource consumption and raises privacy concerns. To this end, we propose a model-agnostic framework for creating representations of activations in DNNs using node-specific histograms to compute p-values of observed activations without retaining already-known inputs. Our proposed approach demonstrates promising potential when validated with multiple network architectures across various downstream tasks and compared with the kernel density estimates and brute-force empirical baselines. In addition, the framework reduces memory usage by 30% with up to 4 times faster p-value computing time while maintaining state of-the-art detection power in downstream tasks such as the detection of adversarial attacks and synthesized content. Moreover, as we do not persist raw data at inference time, we could potentially reduce susceptibility to attacks and privacy issues.