Batch Selection and Communication for Active Learning with Edge Labeling
This addresses communication efficiency in active learning systems, which is incremental as it builds on existing protocols with specific optimizations.
The paper tackles the problem of designing communication protocols for active learning where a learner interacts with a teacher, focusing on batch selection and encoding to reduce communication rounds and resources. It introduces CC-BAKD, a novel protocol combining Bayesian active learning with compression, showing advantages over existing methods in comparisons.
Conventional retransmission (ARQ) protocols are designed with the goal of ensuring the correct reception of all the individual transmitter's packets at the receiver. When the transmitter is a learner communicating with a teacher, this goal is at odds with the actual aim of the learner, which is that of eliciting the most relevant label information from the teacher. Taking an active learning perspective, this paper addresses the following key protocol design questions: (i) Active batch selection: Which batch of inputs should be sent to the teacher to acquire the most useful information and thus reduce the number of required communication rounds? (ii) Batch encoding: Can batches of data points be combined to reduce the communication resources required at each communication round? Specifically, this work introduces Communication-Constrained Bayesian Active Knowledge Distillation (CC-BAKD), a novel protocol that integrates Bayesian active learning with compression via a linear mix-up mechanism. Comparisons with existing active learning protocols demonstrate the advantages of the proposed approach.