Varun Singh

Nikita Tatarinov, Vidhyakshaya Kannan, Haricharana Srinivasa et al. · gatech

The increasing context length of modern language models has created a need for evaluating their ability to retrieve and process information across extensive documents. While existing benchmarks test long-context capabilities, they often lack a structured way to systematically vary question complexity. We introduce KG-QAGen (Knowledge-Graph-based Question-Answer Generation), a framework that (1) extracts QA pairs at multiple complexity levels (2) by leveraging structured representations of financial agreements (3) along three key dimensions -- multi-hop retrieval, set operations, and answer plurality -- enabling fine-grained assessment of model performance across controlled difficulty levels. Using this framework, we construct a dataset of 20,139 QA pairs (the largest number among the long-context benchmarks) and open-source a part of it. We evaluate 13 proprietary and open-source LLMs and observe that even the best-performing models are struggling with set-based comparisons and multi-hop logical inference. Our analysis reveals systematic failure modes tied to semantic misinterpretation and inability to handle implicit relations.

Saba Ahsan, Varun Singh, Jörg Ott

The availability of high definition video content on the web has brought about a significant change in the characteristics of Internet video, but not many studies on characterizing video have been done after this change. Video characteristics such as video length, format, target bit rate, and resolution provide valuable input to design Adaptive Bit Rate (ABR) algorithms, sizing playout buffers in Dynamic Adaptive HTTP streaming (DASH) players, model the variability in video frame sizes, etc. This paper presents datasets collected in 2013 and 2014 that contains over 130,000 videos from YouTube's most viewed (or most popular) video charts in 58 countries. We describe the basic characteristics of the videos on YouTube for each category, format, video length, file size, and data rate variation, observing that video length and file size fit a log normal distribution. We show that three minutes of a video suffice to represent its instant data rate fluctuation and that we can infer data rate characteristics of different video resolutions from a single given one. Based on our findings, we design active measurements for measuring the performance of Internet video.

Marcin Nagy, Varun Singh, Joerg Ott et al.

In this paper, we propose a new rate control algorithm for conversational multimedia flows. In our approach, along with Real-time Transport Protocol (RTP) media packets, we propose sending redundant packets to probe for available bandwidth. These redundant packets are Forward Error Correction (FEC) encoded RTP packets. A straightforward interpretation is that if no losses occur, the sender can increase the sending rate to include the FEC bit rate, and in the case of losses due to congestion the redundant packets help in recovering the lost packets. We also show that by varying the FEC bit rate, the sender is able to conservatively or aggressively probe for available bandwidth. We evaluate our FEC-based Rate Adaptation (FBRA) algorithm in a network simulator and in the real-world and compare it to other congestion control algorithms.