Improving Skin Condition Classification with a Question Answering Model
This work addresses skin condition diagnosis for medical applications, but it is incremental as it builds on existing CNN and QA methods.
The authors tackled skin condition classification by combining a pre-trained CNN with a QA model to improve accuracy and emulate doctor questioning, achieving up to a 10% accuracy increase over the CNN alone and more than 30% over the QA model alone.
We present a skin condition classification methodology based on a sequential pipeline of a pre-trained Convolutional Neural Network (CNN) and a Question Answering (QA) model. This method enables us to not only increase the classification confidence and accuracy of the deployed CNN system, but also enables the emulation of the conventional approach of doctors asking the relevant questions in refining the ultimate diagnosis and differential. By combining the CNN output in the form of classification probabilities as a prior to the QA model and the image textual description, we greedily ask the best symptom that maximizes the information gain over symptoms. We demonstrate that combining the QA model with the CNN increases the accuracy up to 10% as compared to the CNN alone, and more than 30% as compared to the QA model alone.