Sharkzor: Interactive Deep Learning for Image Triage, Sort and Summary
This work addresses the problem of image management for users dealing with large datasets, but it appears incremental as it applies existing deep learning methods to a new interactive interface.
The authors tackled the problem of managing large image collections by developing Sharkzor, a web application that uses deep learning to assist users in triaging, organizing, and summarizing images interactively, resulting in a system that quickly makes sense of large amounts of data through user interaction and automation.
Sharkzor is a web application for machine-learning assisted image sort and summary. Deep learning algorithms are leveraged to infer, augment, and automate the user's mental model. Initially, images uploaded by the user are spread out on a canvas. The user then interacts with the images to impute their mental model into the application's algorithmic underpinnings. Methods of interaction within Sharkzor's user interface and user experience support three primary user tasks; triage, organize and automate. The user triages the large pile of overlapping images by moving images of interest into proximity. The user then organizes said images into meaningful groups. After interacting with the images and groups, deep learning helps to automate the user's interactions. The loop of interaction, automation, and response by the user allows the system to quickly make sense of large amounts of data.