Prannay Khosla

Prannay Khosla, Piotr Teterwak, Chen Wang et al.

Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models. Modern batch contrastive approaches subsume or significantly outperform traditional contrastive losses such as triplet, max-margin and the N-pairs loss. In this work, we extend the self-supervised batch contrastive approach to the fully-supervised setting, allowing us to effectively leverage label information. Clusters of points belonging to the same class are pulled together in embedding space, while simultaneously pushing apart clusters of samples from different classes. We analyze two possible versions of the supervised contrastive (SupCon) loss, identifying the best-performing formulation of the loss. On ResNet-200, we achieve top-1 accuracy of 81.4% on the ImageNet dataset, which is 0.8% above the best number reported for this architecture. We show consistent outperformance over cross-entropy on other datasets and two ResNet variants. The loss shows benefits for robustness to natural corruptions and is more stable to hyperparameter settings such as optimizers and data augmentations. Our loss function is simple to implement, and reference TensorFlow code is released at https://t.ly/supcon.

Prannay Khosla, Moumita Basu, Kripabandhu Ghosh et al.

Microblogging sites like Twitter and Weibo have emerged as important sourcesof real-time information on ongoing events, including socio-political events, emergency events, and so on. For instance, during emergency events (such as earthquakes, floods, terror attacks), microblogging sites are very useful for gathering situational information in real-time. During such an event, typically only a small fraction of the microblogs (tweets) posted are relevant to the information need. Hence, it is necessary to design effective methodologies for microblog retrieval, so that the relevant tweets can be automatically extracted from large sets of documents (tweets). In this work, we apply and compare various neural network-based IR models for microblog retrieval for a specific application, as follows. In a disaster situation, one of the primary and practical challenges in coordinating the post-disaster relief operations is to know about what resources are needed and what resources are available in the disaster-affected area. Thus, in this study, we focus on extracting these two specific types of microblogs or tweets namely need tweets and avail tweets, which are tweets which define some needs of the people and the tweets which offer some solutions or aid for the people, respectively.