Maarten G. Lansberg

Sophie Ostmeier, Brian Axelrod, Jeroen Bertels et al.

Performance metrics for medical image segmentation models are used to measure the agreement between the reference annotation and the predicted segmentation. Usually, overlap metrics, such as the Dice, are used as a metric to evaluate the performance of these models in order for results to be comparable. However, there is a mismatch between the distributions of cases and difficulty level of segmentation tasks in public data sets compared to clinical practice. Common metrics fail to measure the impact of this mismatch, especially for clinical data sets that include low signal pathologies, a difficult segmentation task, and uncertain, small, or empty reference annotations. This limitation may result in ineffective research of machine learning practitioners in designing and optimizing models. Dimensions of evaluating clinical value include consideration of the uncertainty of reference annotations, independence from reference annotation volume size, and evaluation of classification of empty reference annotations. We study how uncertain, small, and empty reference annotations influence the value of metrics for medical image segmentation on an in-house data set regardless of the model. We examine metrics behavior on the predictions of a standard deep learning framework in order to identify metrics with clinical value. We compare to a public benchmark data set (BraTS 2019) with a high-signal pathology and certain, larger, and no empty reference annotations. We may show machine learning practitioners, how uncertain, small, or empty reference annotations require a rethinking of the evaluation and optimizing procedures. The evaluation code was released to encourage further analysis of this topic. https://github.com/SophieOstmeier/UncertainSmallEmpty.git

Yongkai Liu, Helena Feng, Bin Jiang et al.

Predicting outcomes in acute ischemic stroke (AIS) guides clinical decision-making, patient counseling, and resource allocation. Clinical notes contain rich contextual information, but their unstructured nature limits their use in traditional predictive models. We developed and evaluated the Chain-of-Thought (CoT) Outcome Prediction Engine (COPE), a reasoning-enhanced large language model framework, for predicting 90-day functional outcomes after AIS from unstructured clinical notes. This study included 464 AIS patients with discharge summaries and 90-day modified Rankin Scale (mRS) scores. COPE uses a two-step CoT framework based on sequential open-source LLaMA-3-8B models: the first generates clinical reasoning, and the second outputs an mRS prediction. We compared COPE with GPT-4.1, ClinicalBERT, a structured variable-based machine learning model (Clinical ML), and a single-step LLM without CoT. Performance was evaluated using mean absolute error (MAE), accuracy within +/-1 mRS point, and exact accuracy. COPE achieved an MAE of 1.01 (95% CI 0.92-1.11), +/-1 accuracy of 74.4% (69.9, 78.8%), and exact accuracy of 32.8% (28.0, 37.6%), comparable to GPT-4.1 and superior to ClinicalBERT [MAE 1.24 (1.13-1.36)], Clinical ML [1.28 (1.18-1.39)], and the single-step LLM [1.20 (1.09-1.33)]. Subgroup analyses showed consistent performance across sex and age, with slightly higher error among older patients, those undergoing thrombectomy, and those with longer summaries. These findings demonstrate that COPE, a lightweight, interpretable, and privacy-preserving open-source framework, provides an accurate and practical solution for outcome prediction from unstructured clinical text.

Elizabeth D. Vasquez, Jonathan Siskind, Marion S. Buckwalter et al.

Motivation to perform movement therapy and caregiver burnout are major challenges to post-stroke life. Serious games have been shown to support therapeutic tasks in people with stroke, but there are few activities that simultaneously support informal caregiver health, which is also impacted post-stroke. Here, we present a collaborative, mutually beneficial, serious game designed to support therapy for persons with stroke and also exercise for their informal caregivers. One player performs rehabilitative wrist movements - useful to people with stroke - and the other performs a seated march exercise - useful to informal caregivers - via pedals or a keyboard to control their avatar. We present a pilot study with 6 healthy dyads to evaluate how exercise-based input of one player, the Pseudo Caregiver (PCG), impacts motivation and emotional experience in both the PCG and Pseudo Person with Stroke (PPS). While not statistically significant, we find that PCGs Interest subscale scores trended higher when using a pedal (the exercised-based input) compared to a keyboard, regardless of game play mode. PPSs' positive affect scale scores and Competence subscale scores trended higher when their partner played collaboratively with a pedal compared to a keyboard. These trends encourage future work toward incorporating an exercise-based device, such as a pedal, to enhance the emotional and motivational experience of rehabilitative serious games for people with different movement ability levels.

Caitlyn E. Seim, Chuzhang Han, Alexis J. Lowber et al.

Survivors of central nervous system injury commonly present with spastic hypertonia. The affected muscles are hyperexcitable and can display involuntary static muscle tone and an exaggerated stretch reflex. These symptoms affect posture and disrupt activities of daily living. Symptoms are typically measured using subjective manual tests such as the Modified Ashworth Scale; however, more quantitative measures are necessary to evaluate potential treatments. The hands are one of the most common targets for intervention, but few investigators attempt to quantify symptoms of spastic hypertonia affecting the fingers. We present the isometric force pillow (IFP) to quantify involuntary grip force. This lightweight, computerized tool provides a holistic measure of finger flexion force and can be used in various orientations for clinical testing and to measure the impact of assistive devices.