Ilan Shelef

Tal Ben-Haim, Ron Moshe Sofer, Gal Ben-Arie et al.

We present a novel deep learning approach to categorical segmentation of lung CTs of COVID-19 patients. Specifically, we partition the scans into healthy lung tissues, non-lung regions, and two different, yet visually similar, pathological lung tissues, namely, ground-glass opacity and consolidation. This is accomplished via a unique, end-to-end hierarchical network architecture and ensemble learning, which contribute to the segmentation and provide a measure for segmentation uncertainty. The proposed framework achieves competitive results and outstanding generalization capabilities for three COVID-19 datasets. Our method is ranked second in a public Kaggle competition for COVID-19 CT images segmentation. Moreover, segmentation uncertainty regions are shown to correspond to the disagreements between the manual annotations of two different radiologists. Finally, preliminary promising correspondence results are shown for our private dataset when comparing the patients' COVID-19 severity scores (based on clinical measures), and the segmented lung pathologies. Code and data are available at our repository: https://github.com/talbenha/covid-seg

Yisroel Mirsky, Tom Mahler, Ilan Shelef et al.

In 2018, clinics and hospitals were hit with numerous attacks leading to significant data breaches and interruptions in medical services. An attacker with access to medical records can do much more than hold the data for ransom or sell it on the black market. In this paper, we show how an attacker can use deep-learning to add or remove evidence of medical conditions from volumetric (3D) medical scans. An attacker may perform this act in order to stop a political candidate, sabotage research, commit insurance fraud, perform an act of terrorism, or even commit murder. We implement the attack using a 3D conditional GAN and show how the framework (CT-GAN) can be automated. Although the body is complex and 3D medical scans are very large, CT-GAN achieves realistic results which can be executed in milliseconds. To evaluate the attack, we focused on injecting and removing lung cancer from CT scans. We show how three expert radiologists and a state-of-the-art deep learning AI are highly susceptible to the attack. We also explore the attack surface of a modern radiology network and demonstrate one attack vector: we intercepted and manipulated CT scans in an active hospital network with a covert penetration test. Demo video: https://youtu.be/_mkRAArj-x0 Source code: https://github.com/ymirsky/CT-GAN

Or Shwartzman, Harel Gazit, Ilan Shelef et al.

The problem of inter-rater variability is often discussed in the context of manual labeling of medical images. The emergence of data-driven approaches such as Deep Neural Networks (DNNs) brought this issue of raters' disagreement to the front-stage. In this paper, we highlight the issue of inter-rater bias as opposed to random inter-observer variability and demonstrate its influence on DNN training, leading to different segmentation results for the same input images. In fact, lower overlap scores are obtained between the outputs of a DNN trained on annotations of one rater and tested on another. Moreover, we demonstrate that inter-rater bias in the training examples is amplified and becomes more consistent, considering the segmentation predictions of the DNNs' test data. We support our findings by showing that a classifier-DNN trained to distinguish between raters based on their manual annotations performs better when the automatic segmentation predictions rather than the actual raters' annotations were tested. For this study, we used two different datasets: the ISBI 2015 Multiple Sclerosis (MS) challenge dataset, including MRI scans each with annotations provided by two raters with different levels of expertise; and Intracerebral Hemorrhage (ICH) CT scans with manual and semi-manual segmentations. The results obtained allow us to underline a worrisome clinical implication of a DNN bias induced by an inter-rater bias during training. Specifically, we present a consistent underestimate of MS-lesion loads when calculated from segmentation predictions of a DNN trained on input provided by the less experienced rater. In the same manner, the differences in ICH volumes calculated based on outputs of identical DNNs, each trained on annotations from a different source are more consistent and larger than the differences in volumes between the manual and semi-manual annotations used for training.