Interactive Segmentation and Visualization for Tiny Objects in Multi-megapixel Images
This provides a useful research-supporting tool for human-in-the-loop tiny-object segmentation in scientific domains like astronomy, biomedicine, and computer vision, though it is incremental as it integrates existing methods into a new interface.
The paper tackles the cumbersome workflow of detecting cosmic rays in astronomical images by introducing an interactive segmentation and visualization framework that unifies model inference, HDR image visualization, and mask editing into a single graphical user interface, resulting in a browser-based tool with GPU acceleration and multi-user access.
We introduce an interactive image segmentation and visualization framework for identifying, inspecting, and editing tiny objects (just a few pixels wide) in large multi-megapixel high-dynamic-range (HDR) images. Detecting cosmic rays (CRs) in astronomical observations is a cumbersome workflow that requires multiple tools, so we developed an interactive toolkit that unifies model inference, HDR image visualization, segmentation mask inspection and editing into a single graphical user interface. The feature set, initially designed for astronomical data, makes this work a useful research-supporting tool for human-in-the-loop tiny-object segmentation in scientific areas like biomedicine, materials science, remote sensing, etc., as well as computer vision. Our interface features mouse-controlled, synchronized, dual-window visualization of the image and the segmentation mask, a critical feature for locating tiny objects in multi-megapixel images. The browser-based tool can be readily hosted on the web to provide multi-user access and GPU acceleration for any device. The toolkit can also be used as a high-precision annotation tool, or adapted as the frontend for an interactive machine learning framework. Our open-source dataset, CR detection model, and visualization toolkit are available at https://github.com/cy-xu/cosmic-conn.