Multivariate Gaussian Representation Learning for Medical Action Evaluation
This addresses the problem of precise medical action analysis for clinicians, though it appears incremental as it builds on existing spatiotemporal modeling with a novel dataset and method.
The paper tackles fine-grained action evaluation in medical vision by introducing CPREval-6k, a benchmark with 6,372 expert-annotated videos, and proposes GaussMedAct, a multivariate Gaussian encoding framework that achieves 92.1% Top-1 accuracy, outperforming a baseline by +5.9% with only 10% FLOPs.
Fine-grained action evaluation in medical vision faces unique challenges due to the unavailability of comprehensive datasets, stringent precision requirements, and insufficient spatiotemporal dynamic modeling of very rapid actions. To support development and evaluation, we introduce CPREval-6k, a multi-view, multi-label medical action benchmark containing 6,372 expert-annotated videos with 22 clinical labels. Using this dataset, we present GaussMedAct, a multivariate Gaussian encoding framework, to advance medical motion analysis through adaptive spatiotemporal representation learning. Multivariate Gaussian Representation projects the joint motions to a temporally scaled multi-dimensional space, and decomposes actions into adaptive 3D Gaussians that serve as tokens. These tokens preserve motion semantics through anisotropic covariance modeling while maintaining robustness to spatiotemporal noise. Hybrid Spatial Encoding, employing a Cartesian and Vector dual-stream strategy, effectively utilizes skeletal information in the form of joint and bone features. The proposed method achieves 92.1% Top-1 accuracy with real-time inference on the benchmark, outperforming the ST-GCN baseline by +5.9% accuracy with only 10% FLOPs. Cross-dataset experiments confirm the superiority of our method in robustness.