Geoffrey Young

Denis Jered McInerney, Geoffrey Young, Jan-Willem van de Meent et al. · amazon-science, salesforce

We propose CHiLL (Crafting High-Level Latents), an approach for natural-language specification of features for linear models. CHiLL prompts LLMs with expert-crafted queries to generate interpretable features from health records. The resulting noisy labels are then used to train a simple linear classifier. Generating features based on queries to an LLM can empower physicians to use their domain expertise to craft features that are clinically meaningful for a downstream task of interest, without having to manually extract these from raw EHR. We are motivated by a real-world risk prediction task, but as a reproducible proxy, we use MIMIC-III and MIMIC-CXR data and standard predictive tasks (e.g., 30-day readmission) to evaluate this approach. We find that linear models using automatically extracted features are comparably performant to models using reference features, and provide greater interpretability than linear models using "Bag-of-Words" features. We verify that learned feature weights align well with clinical expectations.

Hiba Ahsan, Denis Jered McInerney, Jisoo Kim et al. · amazon-science, salesforce

Unstructured data in Electronic Health Records (EHRs) often contains critical information -- complementary to imaging -- that could inform radiologists' diagnoses. But the large volume of notes often associated with patients together with time constraints renders manually identifying relevant evidence practically infeasible. In this work we propose and evaluate a zero-shot strategy for using LLMs as a mechanism to efficiently retrieve and summarize unstructured evidence in patient EHR relevant to a given query. Our method entails tasking an LLM to infer whether a patient has, or is at risk of, a particular condition on the basis of associated notes; if so, we ask the model to summarize the supporting evidence. Under expert evaluation, we find that this LLM-based approach provides outputs consistently preferred to a pre-LLM information retrieval baseline. Manual evaluation is expensive, so we also propose and validate a method using an LLM to evaluate (other) LLM outputs for this task, allowing us to scale up evaluation. Our findings indicate the promise of LLMs as interfaces to EHR, but also highlight the outstanding challenge posed by "hallucinations". In this setting, however, we show that model confidence in outputs strongly correlates with faithful summaries, offering a practical means to limit confabulations.

Denis Jered McInerney, Geoffrey Young, Jan-Willem van de Meent et al. · amazon-science, salesforce

Pretraining multimodal models on Electronic Health Records (EHRs) provides a means of learning representations that can transfer to downstream tasks with minimal supervision. Recent multimodal models induce soft local alignments between image regions and sentences. This is of particular interest in the medical domain, where alignments might highlight regions in an image relevant to specific phenomena described in free-text. While past work has suggested that attention "heatmaps" can be interpreted in this manner, there has been little evaluation of such alignments. We compare alignments from a state-of-the-art multimodal (image and text) model for EHR with human annotations that link image regions to sentences. Our main finding is that the text has an often weak or unintuitive influence on attention; alignments do not consistently reflect basic anatomical information. Moreover, synthetic modifications -- such as substituting "left" for "right" -- do not substantially influence highlights. Simple techniques such as allowing the model to opt out of attending to the image and few-shot finetuning show promise in terms of their ability to improve alignments with very little or no supervision. We make our code and checkpoints open-source.

Shrikanth Yadav, Jisoo Kim, Geoffrey Young et al.

Background: Computed Tomography Angiography (CTA) is crucial for cerebrovascular disease diagnosis. Dynamic CTA is a type of imaging that captures temporal information about the We aim to develop and evaluate two segmentation techniques to segment vessels directly on CTA images: (1) creating and registering population-averaged vessel atlases and (2) using deep learning (DL). Methods: We retrieved 4D-CT of the head from our institutional research database, with bone and soft tissue subtracted from post-contrast images. An Advanced Normalization Tools pipeline was used to create angiographic atlases from 25 patients. Then, atlas-driven ROIs were identified by a CT attenuation threshold to generate segmentation of the arteries and veins using non-linear registration. To create DL vessel segmentations, arterial and venous structures were segmented using the MRA vessel segmentation tool, iCafe, in 29 patients. These were then used to train a DL model, with bone-in CT images as input. Multiple phase images in the 4D-CT were used to increase the training and validation dataset. Both segmentation approaches were evaluated on a test 4D-CT dataset of 11 patients which were also processed by iCafe and validated by a neuroradiologist. Specifically, branch-wise segmentation accuracy was quantified with 20 labels for arteries and one for veins. DL outperformed the atlas-based segmentation models for arteries (average modified dice coefficient (amDC) 0.856 vs. 0.324) and veins (amDC 0.743 vs. 0.495) overall. For ICAs, vertebral and basilar arteries, DL and atlas -based segmentation had an amDC of 0.913 and 0.402, respectively. The amDC for MCA-M1, PCA-P1, and ACA-A1 segments were 0.932 and 0.474, respectively. Conclusion: Angiographic CT templates are developed for the first time in literature. Using 4D-CTA enables the use of tools like iCafe, lessening the burden of manual annotation.

Denis Jered McInerney, Borna Dabiri, Anne-Sophie Touret et al.

Electronic Health Records (EHRs) provide vital contextual information to radiologists and other physicians when making a diagnosis. Unfortunately, because a given patient's record may contain hundreds of notes and reports, identifying relevant information within these in the short time typically allotted to a case is very difficult. We propose and evaluate models that extract relevant text snippets from patient records to provide a rough case summary intended to aid physicians considering one or more diagnoses. This is hard because direct supervision (i.e., physician annotations of snippets relevant to specific diagnoses in medical records) is prohibitively expensive to collect at scale. We propose a distantly supervised strategy in which we use groups of International Classification of Diseases (ICD) codes observed in 'future' records as noisy proxies for 'downstream' diagnoses. Using this we train a transformer-based neural model to perform extractive summarization conditioned on potential diagnoses. This model defines an attention mechanism that is conditioned on potential diagnoses (queries) provided by the diagnosing physician. We train (via distant supervision) and evaluate variants of this model on EHR data from Brigham and Women's Hospital in Boston and MIMIC-III (the latter to facilitate reproducibility). Evaluations performed by radiologists demonstrate that these distantly supervised models yield better extractive summaries than do unsupervised approaches. Such models may aid diagnosis by identifying sentences in past patient reports that are clinically relevant to a potential diagnosis.