Menglin Li

Yu-Jen Chen, Wei-Hsiang Shen, Hao-Wei Chung et al. · pku

Medical report generation is a challenging task since it is time-consuming and requires expertise from experienced radiologists. The goal of medical report generation is to accurately capture and describe the image findings. Previous works pretrain their visual encoding neural networks with large datasets in different domains, which cannot learn general visual representation in the specific medical domain. In this work, we propose a medical report generation framework that uses a contrastive learning approach to pretrain the visual encoder and requires no additional meta information. In addition, we adopt lung segmentation as an augmentation method in the contrastive learning framework. This segmentation guides the network to focus on encoding the visual feature within the lung region. Experimental results show that the proposed framework improves the performance and the quality of the generated medical reports both quantitatively and qualitatively.

Menglin Li, Kwan Hui Lim, Teng Guo et al.

POI-level geo-information of social posts is critical to many location-based applications and services. However, the multi-modality, complexity and diverse nature of social media data and their platforms limit the performance of inferring such fine-grained locations and their subsequent applications. To address this issue, we present a transformer-based general framework, which builds upon pre-trained language models and considers non-textual data, for social post geolocation at the POI level. To this end, inputs are categorized to handle different social data, and an optimal combination strategy is provided for feature representations. Moreover, a uniform representation of hierarchy is proposed to learn temporal information, and a concatenated version of encodings is employed to capture feature-wise positions better. Experimental results on various social datasets demonstrate that three variants of our proposed framework outperform multiple state-of-art baselines by a large margin in terms of accuracy and distance error metrics.

Junhua Liu, Kwan Hui Lim, Kristin L. Wood et al.

Itinerary recommendation is a complex sequence prediction problem with numerous real-world applications. This task becomes even more challenging when considering the optimization of multiple user queuing times and crowd levels, as well as numerous involved parameters, such as attraction popularity, queuing time, walking time, and operating hours. Existing solutions typically focus on single-person perspectives and fail to address real-world issues resulting from natural crowd behavior, like the Selfish Routing problem. In this paper, we introduce the Strategic and Crowd-Aware Itinerary Recommendation (SCAIR) algorithm, which optimizes group utility in real-world settings. We model the route recommendation strategy as a Markov Decision Process and propose a State Encoding mechanism that enables real-time planning and allocation in linear time. We evaluate our algorithm against various competitive and realistic baselines using a theme park dataset, demonstrating that SCAIR outperforms these baselines in addressing the Selfish Routing problem across four theme parks.

Menglin Li, Kwan Hui Lim

To address the challenges of scarcity in geotagged data for social user geolocation, we propose FewUser, a novel framework for Few-shot social User geolocation. We incorporate a contrastive learning strategy between users and locations to improve geolocation performance with no or limited training data. FewUser features a user representation module that harnesses a pre-trained language model (PLM) and a user encoder to process and fuse diverse social media inputs effectively. To bridge the gap between PLM's knowledge and geographical data, we introduce a geographical prompting module with hard, soft, and semi-soft prompts, to enhance the encoding of location information. Contrastive learning is implemented through a contrastive loss and a matching loss, complemented by a hard negative mining strategy to refine the learning process. We construct two datasets TwiU and FliU, containing richer metadata than existing benchmarks, to evaluate FewUser and the extensive experiments demonstrate that FewUser significantly outperforms state-of-the-art methods in both zero-shot and various few-shot settings, achieving absolute improvements of 26.95\% and \textbf{41.62\%} on TwiU and FliU, respectively, with only one training sample per class. We further conduct a comprehensive analysis to investigate the impact of user representation on geolocation performance and the effectiveness of FewUser's components, offering valuable insights for future research in this area.