Investigating the Robustness and Properties of Detection Transformers (DETR) Toward Difficult Images
This work addresses robustness issues in transformer-based object detectors for computer vision applications, but it is incremental as it benchmarks against existing methods without proposing new solutions.
The paper investigated how Detection Transformers (DETR) handle difficult images like occlusions and adversarial perturbations, finding that DETR resists occlusion well but performs worse than YOLOv5 on image corruption and is vulnerable to adversarial stickers due to query misdirection.
Transformer-based object detectors (DETR) have shown significant performance across machine vision tasks, ultimately in object detection. This detector is based on a self-attention mechanism along with the transformer encoder-decoder architecture to capture the global context in the image. The critical issue to be addressed is how this model architecture can handle different image nuisances, such as occlusion and adversarial perturbations. We studied this issue by measuring the performance of DETR with different experiments and benchmarking the network with convolutional neural network (CNN) based detectors like YOLO and Faster-RCNN. We found that DETR performs well when it comes to resistance to interference from information loss in occlusion images. Despite that, we found that the adversarial stickers put on the image require the network to produce a new unnecessary set of keys, queries, and values, which in most cases, results in a misdirection of the network. DETR also performed poorer than YOLOv5 in the image corruption benchmark. Furthermore, we found that DETR depends heavily on the main query when making a prediction, which leads to imbalanced contributions between queries since the main query receives most of the gradient flow.