Reducing Overlearning through Disentangled Representations by Suppressing Unknown Tasks
This addresses privacy preservation in AI by preventing models from learning unintended tasks, though it appears incremental as it builds on existing disentanglement and suppression concepts.
The paper tackles the problem of deep learning models overlearning and extracting more information than needed for a task, proposing a model-agnostic solution to suppress unknown tasks without requiring ground truth labels, and demonstrates results on datasets like PreserveTask and color-MNIST with practical applications in face attribute preservation.
Existing deep learning approaches for learning visual features tend to overlearn and extract more information than what is required for the task at hand. From a privacy preservation perspective, the input visual information is not protected from the model; enabling the model to become more intelligent than it is trained to be. Current approaches for suppressing additional task learning assume the presence of ground truth labels for the tasks to be suppressed during training time. In this research, we propose a three-fold novel contribution: (i) a model-agnostic solution for reducing model overlearning by suppressing all the unknown tasks, (ii) a novel metric to measure the trust score of a trained deep learning model, and (iii) a simulated benchmark dataset, PreserveTask, having five different fundamental image classification tasks to study the generalization nature of models. In the first set of experiments, we learn disentangled representations and suppress overlearning of five popular deep learning models: VGG16, VGG19, Inception-v1, MobileNet, and DenseNet on PreserverTask dataset. Additionally, we show results of our framework on color-MNIST dataset and practical applications of face attribute preservation in Diversity in Faces (DiF) and IMDB-Wiki dataset.