Iterative Teaching by Data Hallucination
This addresses the problem of enhancing teaching capabilities in machine learning for scenarios with limited data pools, though it appears incremental as it builds on existing iterative teaching frameworks.
The paper tackles the limitation of iterative machine teaching in discrete input spaces by proposing data hallucination teaching (DHT), which allows intelligent generation of input data in continuous spaces, and shows effectiveness in various challenging setups like linear and neural learners.
We consider the problem of iterative machine teaching, where a teacher sequentially provides examples based on the status of a learner under a discrete input space (i.e., a pool of finite samples), which greatly limits the teacher's capability. To address this issue, we study iterative teaching under a continuous input space where the input example (i.e., image) can be either generated by solving an optimization problem or drawn directly from a continuous distribution. Specifically, we propose data hallucination teaching (DHT) where the teacher can generate input data intelligently based on labels, the learner's status and the target concept. We study a number of challenging teaching setups (e.g., linear/neural learners in omniscient and black-box settings). Extensive empirical results verify the effectiveness of DHT.