Two-Stream Deep Feature Modelling for Automated Video Endoscopy Data Analysis
This work addresses diagnostic support for medical practitioners in GI endoscopy, but it is incremental as it builds on existing deep learning methods with a specific architectural improvement.
The authors tackled automated analysis of gastrointestinal endoscopy images by proposing a two-stream deep feature model with a novel relational network to fuse streams, achieving state-of-the-art performance on KVASIR and Nerthus datasets.
Automating the analysis of imagery of the Gastrointestinal (GI) tract captured during endoscopy procedures has substantial potential benefits for patients, as it can provide diagnostic support to medical practitioners and reduce mistakes via human error. To further the development of such methods, we propose a two-stream model for endoscopic image analysis. Our model fuses two streams of deep feature inputs by mapping their inherent relations through a novel relational network model, to better model symptoms and classify the image. In contrast to handcrafted feature-based models, our proposed network is able to learn features automatically and outperforms existing state-of-the-art methods on two public datasets: KVASIR and Nerthus. Our extensive evaluations illustrate the importance of having two streams of inputs instead of a single stream and also demonstrates the merits of the proposed relational network architecture to combine those streams.