Personalizing Federated Instrument Segmentation with Visual Trait Priors in Robotic Surgery
This work addresses the challenge of improving segmentation accuracy in robotic surgery across multiple clinical sites while preserving privacy, representing an incremental advance in personalized federated learning.
The paper tackles the problem of personalized federated learning for surgical instrument segmentation by proposing PFedSIS, a method that incorporates visual trait priors to address appearance diversity and shape similarity, resulting in performance gains of +1.51% Dice, +2.11% IoU, -2.79 ASSD, and -15.55 HD95 over state-of-the-art methods.
Personalized federated learning (PFL) for surgical instrument segmentation (SIS) is a promising approach. It enables multiple clinical sites to collaboratively train a series of models in privacy, with each model tailored to the individual distribution of each site. Existing PFL methods rarely consider the personalization of multi-headed self-attention, and do not account for appearance diversity and instrument shape similarity, both inherent in surgical scenes. We thus propose PFedSIS, a novel PFL method with visual trait priors for SIS, incorporating global-personalized disentanglement (GPD), appearance-regulation personalized enhancement (APE), and shape-similarity global enhancement (SGE), to boost SIS performance in each site. GPD represents the first attempt at head-wise assignment for multi-headed self-attention personalization. To preserve the unique appearance representation of each site and gradually leverage the inter-site difference, APE introduces appearance regulation and provides customized layer-wise aggregation solutions via hypernetworks for each site's personalized parameters. The mutual shape information of instruments is maintained and shared via SGE, which enhances the cross-style shape consistency on the image level and computes the shape-similarity contribution of each site on the prediction level for updating the global parameters. PFedSIS outperforms state-of-the-art methods with +1.51% Dice, +2.11% IoU, -2.79 ASSD, -15.55 HD95 performance gains. The corresponding code and models will be released at https://github.com/wzjialang/PFedSIS.