Does Explainable Machine Learning Uncover the Black Box in Vision Applications?
This is an incremental critique for researchers and practitioners in computer vision and explainable AI, highlighting gaps in existing methods.
The paper argues that current explainable machine learning approaches in vision applications have limitations and fail to meaningfully uncover black box models, raising fundamental questions and suggesting reliance on more rigorous principles.
Machine learning (ML) in general and deep learning (DL) in particular has become an extremely popular tool in several vision applications (like object detection, super resolution, segmentation, object tracking etc.). Almost in parallel, the issue of explainability in ML (i.e. the ability to explain/elaborate the way a trained ML model arrived at its decision) in vision has also received fairly significant attention from various quarters. However, we argue that the current philosophy behind explainable ML suffers from certain limitations, and the resulting explanations may not meaningfully uncover black box ML models. To elaborate our assertion, we first raise a few fundamental questions which have not been adequately discussed in the corresponding literature. We also provide perspectives on how explainablity in ML can benefit by relying on more rigorous principles in the related areas.