Yu-Yun Tseng

Yu-Yun Tseng, Alexander Bell, Danna Gurari

We introduce a few-shot localization dataset originating from photographers who authentically were trying to learn about the visual content in the images they took. It includes nearly 10,000 segmentations of 100 categories in over 4,500 images that were taken by people with visual impairments. Compared to existing few-shot object detection and instance segmentation datasets, our dataset is the first to locate holes in objects (e.g., found in 12.3\% of our segmentations), it shows objects that occupy a much larger range of sizes relative to the images, and text is over five times more common in our objects (e.g., found in 22.4\% of our segmentations). Analysis of three modern few-shot localization algorithms demonstrates that they generalize poorly to our new dataset. The algorithms commonly struggle to locate objects with holes, very small and very large objects, and objects lacking text. To encourage a larger community to work on these unsolved challenges, we publicly share our annotated few-shot dataset at https://vizwiz.org .

Yu-Yun Tseng, Tanusree Sharma, Lotus Zhang et al.

Individuals who are blind or have low vision (BLV) are at a heightened risk of sharing private information if they share photographs they have taken. To facilitate developing technologies that can help them preserve privacy, we introduce BIV-Priv-Seg, the first localization dataset originating from people with visual impairments that shows private content. It contains 1,028 images with segmentation annotations for 16 private object categories. We first characterize BIV-Priv-Seg and then evaluate modern models' performance for locating private content in the dataset. We find modern models struggle most with locating private objects that are not salient, small, and lack text as well as recognizing when private content is absent from an image. We facilitate future extensions by sharing our new dataset with the evaluation server at https://vizwiz.org/tasks-and-datasets/object-localization.

Tanusree Sharma, Yu-Yun Tseng, Lotus Zhang et al.

Blind and low vision (BLV) individuals use Generative AI (GenAI) tools to interpret and manage visual content in their daily lives. While such tools can enhance the accessibility of visual content and so enable greater user independence, they also introduce complex challenges around visual privacy. In this paper, we investigate the current practices and future design preferences of blind and low vision individuals through an interview study with 21 participants. Our findings reveal a range of current practices with GenAI that balance privacy, efficiency, and emotional agency, with users accounting for privacy risks across six key scenarios, such as self-presentation, indoor/outdoor spatial privacy, social sharing, and handling professional content. Our findings reveal design preferences, including on-device processing, zero-retention guarantees, sensitive content redaction, privacy-aware appearance indicators, and multimodal tactile mirrored interaction methods. We conclude with actionable design recommendations to support user-centered visual privacy through GenAI, expanding the notion of privacy and responsible handling of others data.

Chongyan Chen, Yu-Yun Tseng, Zhuoheng Li et al.

No published work on visual question answering (VQA) accounts for ambiguity regarding where the content described in the question is located in the image. To fill this gap, we introduce VQ-FocusAmbiguity, the first VQA dataset that visually grounds each plausible image region a question could refer to when arriving at valid answers. We next analyze and compare our dataset to existing datasets to reveal its unique properties. Finally, we benchmark modern models for two novel tasks related to acknowledging focus ambiguity: recognizing whether a visual question has focus ambiguity and locating all plausible focus regions within the image. Results show that the dataset is challenging for modern models. To facilitate future progress on these tasks, we publicly share the dataset with an evaluation server at https://vizwiz.org/tasks-and-datasets/focus-ambiguity-in-visual-questions.

Yu Chan, Pin-Yu Lin, Yu-Yun Tseng et al.

WiFi-based indoor positioning has been extensively studied. A fundamental issue in such solutions is the collection of WiFi fingerprints. However, due to real-world constraints, collecting complete fingerprints at all intended locations is sometimes prohibited. This work considers the WiFi fingerprint inpainting problem. This problem differs from typical image/video inpainting problems in several aspects. Unlike RGB images, WiFi field maps come in any shape, and signal data may follow certain distributions. Therefore, it is difficult to forcefully fit them into a fixed-dimensional matrix, as done with processing images in RGB format. As soon as a map is changed, it also becomes difficult to adapt it to the same model due to scale issues. Furthermore, such models are significantly constrained in situations requiring outward inpainting. Fortunately, the spatial relationships of WiFi signals and the rich information provided among channels offer ample opportunities for this generative model to accomplish inpainting. Therefore, we designed this model to not only retain the characteristic of regression models in generating fingerprints of arbitrary shapes but also to accommodate the observational outcomes from densely deployed APs. This work makes two major contributions. Firstly, we delineate the distinctions between this problem and image inpainting, highlighting potential avenues for research. Secondly, we introduce novel generative inpainting models aimed at capturing both inter-AP and intra-AP correlations while preserving latent information. Additionally, we incorporate a specially designed adversarial discriminator to enhance the quality of inpainting outcomes.