Osman Yağan

Ishank Juneja, Carlee Joe-Wong, Osman Yağan

The classic multi-armed bandit (MAB) problem tackles the challenge of accruing maximum reward while making decisions under uncertainty. However, in applications, often the goal is to minimize cost subject to a constraint on the minimum permissible reward, an objective captured by multi-armed bandits with cost-subsidy (MAB-CS). Of interest to this paper is the setting where the quality (reward) constraint is specified relative to the unknown best reward and the cost of each arm is known. We characterize the expected sub-optimal samples required by any policy by proving instance-dependent lower bounds that offer new insight into the problem and are a strict generalization of prior bounds. Then, we propose an algorithm called Cost-Ordered Feasibility (COF) that leverages our insight and intelligently combine samples from all arms to gauge the feasibility of a cheap arm. Thereafter, we analyze COF to establish instance-dependent upper bounds on its expected cumulative cost and quality regret, i.e., relative to the cheapest feasible arm. Finally, we empirically validate the merits of COF, comparing it to baselines from the literature through extensive simulation experiments on the MovieLens and Goodreads datasets as well as representative synthetic instances. Not only does our paper develop qualitatively better theoretical regret upper bounds, but COF also convincingly demonstrates improved empirical performance.

Ishank Juneja, Carlee Joe-Wong, Osman Yağan

Multi-armed bandits (MAB) are commonly used in sequential online decision-making when the reward of each decision is an unknown random variable. In practice however, the typical goal of maximizing total reward may be less important than minimizing the total cost of the decisions taken, subject to a reward constraint. For example, we may seek to make decisions that have at least the reward of a reference ``default'' decision, with as low a cost as possible. This problem was recently introduced in the Multi-Armed Bandits with Cost Subsidy (MAB-CS) framework. MAB-CS is broadly applicable to problem domains where a primary metric (cost) is constrained by a secondary metric (reward), and the rewards are unknown. In our work, we address variants of MAB-CS including ones with reward constrained by the reward of a known reference arm or by the subsidized best reward. We introduce the Pairwise-Elimination (PE) algorithm for the known reference arm variant and generalize PE to PE-CS for the subsidized best reward variant. Our instance-dependent analysis of PE and PE-CS reveals that both algorithms have an order-wise logarithmic upper bound on Cost and Quality Regret, making our policies the first with such a guarantee. Moreover, by comparing our upper and lower bound results we establish that PE is order-optimal for all known reference arm problem instances. Finally, experiments are conducted using the MovieLens 25M and Goodreads datasets for both PE and PE-CS revealing the effectiveness of PE and the superior balance between performance and reliability offered by PE-CS compared to baselines from the literature.

Samarth Gupta, Gauri Joshi, Osman Yağan

In this paper we consider the problem of best-arm identification in multi-armed bandits in the fixed confidence setting, where the goal is to identify, with probability $1-δ$ for some $δ>0$, the arm with the highest mean reward in minimum possible samples from the set of arms $\mathcal{K}$. Most existing best-arm identification algorithms and analyses operate under the assumption that the rewards corresponding to different arms are independent of each other. We propose a novel correlated bandit framework that captures domain knowledge about correlation between arms in the form of upper bounds on expected conditional reward of an arm, given a reward realization from another arm. Our proposed algorithm C-LUCB, which generalizes the LUCB algorithm utilizes this partial knowledge of correlations to sharply reduce the sample complexity of best-arm identification. More interestingly, we show that the total samples obtained by C-LUCB are of the form $\mathcal{O}\left(\sum_{k \in \mathcal{C}} \log\left(\frac{1}δ\right)\right)$ as opposed to the typical $\mathcal{O}\left(\sum_{k \in \mathcal{K}} \log\left(\frac{1}δ\right)\right)$ samples required in the independent reward setting. The improvement comes, as the $\mathcal{O}(\log(1/δ))$ term is summed only for the set of competitive arms $\mathcal{C}$, which is a subset of the original set of arms $\mathcal{K}$. The size of the set $\mathcal{C}$, depending on the problem setting, can be as small as $2$, and hence using C-LUCB in the correlated bandits setting can lead to significant performance improvements. Our theoretical findings are supported by experiments on the Movielens and Goodreads recommendation datasets.

Yae Jee Cho, Samarth Gupta, Gauri Joshi et al.

Due to communication constraints and intermittent client availability in federated learning, only a subset of clients can participate in each training round. While most prior works assume uniform and unbiased client selection, recent work on biased client selection has shown that selecting clients with higher local losses can improve error convergence speed. However, previously proposed biased selection strategies either require additional communication cost for evaluating the exact local loss or utilize stale local loss, which can even make the model diverge. In this paper, we present a bandit-based communication-efficient client selection strategy UCB-CS that achieves faster convergence with lower communication overhead. We also demonstrate how client selection can be used to improve fairness.

Mansi Sood, Osman Yağan

Random graphs are an important tool for modelling and analyzing the underlying properties of complex real-world networks. In this paper, we study a class of random graphs known as the inhomogeneous random K-out graphs which were recently introduced to analyze heterogeneous sensor networks secured by the pairwise scheme. In this model, first, each of the $n$ nodes is classified as type-1 (respectively, type-2) with probability $0<μ<1$ (respectively, $1-μ)$ independently from each other. Next, each type-1 (respectively, type-2) node draws 1 arc towards a node (respectively, $K_n$ arcs towards $K_n$ distinct nodes) selected uniformly at random, and then the orientation of the arcs is ignored. From the literature on homogeneous K-out graphs wherein all nodes select $K_n$ neighbors (i.e., $μ=0$), it is known that when $K_n \geq2$, the graph is $K_n$-connected asymptotically almost surely (a.a.s.) as $n$ gets large. In the inhomogeneous case (i.e., $μ>0$), it was recently established that achieving even 1-connectivity a.a.s. requires $K_n=ω(1)$. Here, we provide a comprehensive set of results to complement these existing results. First, we establish a sharp zero-one law for $k$-connectivity, showing that for the network to be $k$-connected a.a.s., we need to set $K_n = \frac{1}{1-μ}(\log n +(k-2)\log\log n + ω(1))$ for all $k=2, 3, \ldots$. Despite such large scaling of $K_n$ being required for $k$-connectivity, we show that the trivial condition of $K_n \geq 2$ for all $n$ is sufficient to ensure that inhomogeneous K-out graph has a connected component of size $n-O(1)$ whp.

Samarth Gupta, Shreyas Chaudhari, Gauri Joshi et al.

We consider a multi-armed bandit framework where the rewards obtained by pulling different arms are correlated. We develop a unified approach to leverage these reward correlations and present fundamental generalizations of classic bandit algorithms to the correlated setting. We present a unified proof technique to analyze the proposed algorithms. Rigorous analysis of C-UCB (the correlated bandit version of Upper-confidence-bound) reveals that the algorithm ends up pulling certain sub-optimal arms, termed as non-competitive, only O(1) times, as opposed to the O(log T) pulls required by classic bandit algorithms such as UCB, TS etc. We present regret-lower bound and show that when arms are correlated through a latent random source, our algorithms obtain order-optimal regret. We validate the proposed algorithms via experiments on the MovieLens and Goodreads datasets, and show significant improvement over classical bandit algorithms.

Samarth Gupta, Shreyas Chaudhari, Subhojyoti Mukherjee et al.

We consider a finite-armed structured bandit problem in which mean rewards of different arms are known functions of a common hidden parameter $θ^*$. Since we do not place any restrictions of these functions, the problem setting subsumes several previously studied frameworks that assume linear or invertible reward functions. We propose a novel approach to gradually estimate the hidden $θ^*$ and use the estimate together with the mean reward functions to substantially reduce exploration of sub-optimal arms. This approach enables us to fundamentally generalize any classic bandit algorithm including UCB and Thompson Sampling to the structured bandit setting. We prove via regret analysis that our proposed UCB-C algorithm (structured bandit versions of UCB) pulls only a subset of the sub-optimal arms $O(\log T)$ times while the other sub-optimal arms (referred to as non-competitive arms) are pulled $O(1)$ times. As a result, in cases where all sub-optimal arms are non-competitive, which can happen in many practical scenarios, the proposed algorithms achieve bounded regret. We also conduct simulations on the Movielens recommendations dataset to demonstrate the improvement of the proposed algorithms over existing structured bandit algorithms.

Samarth Gupta, Gauri Joshi, Osman Yağan

We consider a novel multi-armed bandit framework where the rewards obtained by pulling the arms are functions of a common latent random variable. The correlation between arms due to the common random source can be used to design a generalized upper-confidence-bound (UCB) algorithm that identifies certain arms as $non-competitive$, and avoids exploring them. As a result, we reduce a $K$-armed bandit problem to a $C+1$-armed problem, where $C+1$ includes the best arm and $C$ $competitive$ arms. Our regret analysis shows that the competitive arms need to be pulled $\mathcal{O}(\log T)$ times, while the non-competitive arms are pulled only $\mathcal{O}(1)$ times. As a result, there are regimes where our algorithm achieves a $\mathcal{O}(1)$ regret as opposed to the typical logarithmic regret scaling of multi-armed bandit algorithms. We also evaluate lower bounds on the expected regret and prove that our correlated-UCB algorithm achieves $\mathcal{O}(1)$ regret whenever possible.

Samarth Gupta, Gauri Joshi, Osman Yağan

This paper studies the problem of {\em learning} the probability distribution $P_X$ of a discrete random variable $X$ using indirect and sequential samples. At each time step, we choose one of the possible $K$ functions, $g_1, \ldots, g_K$ and observe the corresponding sample $g_i(X)$. The goal is to estimate the probability distribution of $X$ by using a minimum number of such sequential samples. This problem has several real-world applications including inference under non-precise information and privacy-preserving statistical estimation. We establish necessary and sufficient conditions on the functions $g_1, \ldots, g_K$ under which asymptotically consistent estimation is possible. We also derive lower bounds on the estimation error as a function of total samples and show that it is order-wise achievable. Leveraging these results, we propose an iterative algorithm that i) chooses the function to observe at each step based on past observations; and ii) combines the obtained samples to estimate $p_X$. The performance of this algorithm is investigated numerically under various scenarios, and shown to outperform baseline approaches.

Rashad Eletreby, Osman Yağan

We consider secure and reliable connectivity in wireless sensor networks that utilize a heterogeneous random key predistribution scheme. We model the unreliability of wireless links by an on-off channel model that induces an Erdős-Rényi graph, while the heterogeneous scheme induces an inhomogeneous random key graph. The overall network can thus be modeled by the intersection of both graphs. We present conditions (in the form of zero-one laws) on how to scale the parameters of the intersection model so that with high probability i) all of its nodes are connected to at least $k$ other nodes; i.e., the minimum node degree of the graph is no less than $k$ and ii) the graph is $k$-connected, i.e., the graph remains connected even if any $k-1$ nodes leave the network. We also present numerical results to support these conditions in the finite-node regime. Our results are shown to complement and generalize several previous work in the literature.

Rashad Eletreby, Osman Yağan

We consider wireless sensor networks under a heterogeneous random key predistribution scheme and an on-off channel model. The heterogeneous key predistribution scheme has recently been introduced by Yağan - as an extension to the Eschenauer and Gligor scheme - for the cases when the network consists of sensor nodes with varying level of resources and/or connectivity requirements, e.g., regular nodes vs. cluster heads. The network is modeled by the intersection of the inhomogeneous random key graph (induced by the heterogeneous scheme) with an Erdős-Rényi graph (induced by the on/off channel model). We present conditions (in the form of zero-one laws) on how to scale the parameters of the intersection model so that with high probability all of its nodes are connected to at least $k$ other nodes; i.e., the minimum node degree of the graph is no less than $k$. We also present numerical results to support our results in the finite-node regime. The numerical results suggest that the conditions that ensure $k$-connectivity coincide with those ensuring the minimum node degree being no less than $k$.

Jun Zhao, Osman Yağan, Virgil Gligor

In wireless sensor networks (WSNs), the Eschenauer-Gligor (EG) key pre-distribution scheme is a widely recognized way to secure communications. Although connectivity properties of secure WSNs with the EG scheme have been extensively investigated, few results address physical transmission constraints. These constraints reflect real-world implementations of WSNs in which two sensors have to be within a certain distance from each other to communicate. In this paper, we present zero-one laws for connectivity in WSNs employing the EG scheme under transmission constraints. These laws help specify the critical transmission ranges for connectivity. Our analytical findings are confirmed via numerical experiments. In addition to secure WSNs, our theoretical results are also applied to frequency hopping in wireless networks.

Jun Zhao, Osman Yağan, Virgil Gligor

Random intersection graphs have received much interest and been used in diverse applications. They are naturally induced in modeling secure sensor networks under random key predistribution schemes, as well as in modeling the topologies of social networks including common-interest networks, collaboration networks, and actor networks. Simply put, a random intersection graph is constructed by assigning each node a set of items in some random manner and then putting an edge between any two nodes that share a certain number of items. Broadly speaking, our work is about analyzing random intersection graphs, and models generated by composing it with other random graph models including random geometric graphs and Erdős-Rényi graphs. These compositional models are introduced to capture the characteristics of various complex natural or man-made networks more accurately than the existing models in the literature. For random intersection graphs and their compositions with other random graphs, we study properties such as ($k$-)connectivity, ($k$-)robustness, and containment of perfect matchings and Hamilton cycles. Our results are typically given in the form of asymptotically exact probabilities or zero-one laws specifying critical scalings, and provide key insights into the design and analysis of various real-world networks.

Jun Zhao, Osman Yağan, Virgil Gligor

In wireless sensor networks, the $q$-composite key predistribution scheme is a widely recognized way to secure communications. Although connectivity properties of secure sensor networks with the $q$-composite scheme have been studied in the literature, few results address physical transmission constraints since it is challenging to analyze the network connectivity in consideration of both the $q$-composite scheme and transmission constraints together. These transmission constraints reflect real-world implementations of sensor networks in which two sensors have to be within a certain distance from each other to communicate. In this paper, we rigorously derive conditions for connectivity in sensor networks employing the $q$-composite scheme under transmission constraints. Furthermore, we extend the analysis to consider the unreliability of wireless links by modeling each link being independently active with some probability. Our results provide useful guidelines for designing securely and reliably connected sensor networks. We also present numerical experiments to confirm the analytical results.

Jun Zhao, Osman Yağan, Virgil Gligor

The q-composite key predistribution scheme [1] is used prevalently for secure communications in large-scale wireless sensor networks (WSNs). Prior work [2]-[4] explores topological properties of WSNs employing the q-composite scheme for q = 1 with unreliable communication links modeled as independent on/off channels. In this paper, we investigate topological properties related to the node degree in WSNs operating under the q-composite scheme and the on/off channel model. Our results apply to general q and are stronger than those reported for the node degree in prior work even for the case of q being 1. Specifically, we show that the number of nodes with certain degree asymptotically converges in distribution to a Poisson random variable, present the asymptotic probability distribution for the minimum degree of the network, and establish the asymptotically exact probability for the property that the minimum degree is at least an arbitrary value. Numerical experiments confirm the validity of our analytical findings.

Faruk Yavuz, Jun Zhao, Osman Yağan et al.

We study the secure and reliable connectivity of wireless sensor networks. Security is assumed to be ensured by the random pairwise key predistribution scheme of Chan, Perrig, and Song, and unreliable wireless links are represented by independent on/off channels. Modeling the network by an intersection of a random $K$-out graph and an Erdős-Rényi graph, we present scaling conditions (on the number of nodes, the scheme parameter $K$, and the probability of a wireless channel being on) such that the resulting graph contains no nodes with degree less than $k$ with high probability, when the number of nodes gets large. Results are given in the form of zero-one laws and are shown to improve the previous results by Yağan and Makowski on the absence of isolated nodes (i.e., absence of nodes with degree zero). Via simulations, the established zero-one laws are shown to hold also for the property of $k$-connectivity; i.e., the property that graph remains connected despite the deletion of any $k-1$ nodes or edges.