Phillip Smith

Venelin Kovatchev, Phillip Smith, Mark Lee et al.

In this paper we implement and compare 7 different data augmentation strategies for the task of automatic scoring of children's ability to understand others' thoughts, feelings, and desires (or "mindreading"). We recruit in-domain experts to re-annotate augmented samples and determine to what extent each strategy preserves the original rating. We also carry out multiple experiments to measure how much each augmentation strategy improves the performance of automatic scoring systems. To determine the capabilities of automatic systems to generalize to unseen data, we create UK-MIND-20 - a new corpus of children's performance on tests of mindreading, consisting of 10,320 question-answer pairs. We obtain a new state-of-the-art performance on the MIND-CA corpus, improving macro-F1-score by 6 points. Results indicate that both the number of training examples and the quality of the augmentation strategies affect the performance of the systems. The task-specific augmentations generally outperform task-agnostic augmentations. Automatic augmentations based on vectors (GloVe, FastText) perform the worst. We find that systems trained on MIND-CA generalize well to UK-MIND-20. We demonstrate that data augmentation strategies also improve the performance on unseen data.

Phillip Smith, Anh Luong, Shamik Sarkar et al.

Software-defined radios (SDRs) are often used in the experimental evaluation of next-generation wireless technologies. While crowd-sourced spectrum monitoring is an important component of future spectrum-agile technologies, there is no clear way to test it in the real world, i.e., with hundreds of users each with an SDR in their pocket participating in RF experiments controlled by, and data uploaded to, the cloud. Current fully functional SDRs are bulky, with components connected via wires, and last at most hours on a single battery charge. To address the needs of such experiments, we design and develop a compact, portable, untethered, and inexpensive SDR we call Sitara. Our SDR interfaces with a mobile device over Bluetooth 5 and can function standalone or as a client to a central command and control server. The Sitara offers true portability: it operates up to one week on battery power, requires no external wired connections and occupies a footprint smaller than a credit card. It transmits and receives common waveforms, uploads IQ samples or processed receiver data through a mobile device to a server for remote processing and performs spectrum sensing functions. Multiple Sitaras form a distributed system capable of conducting experiments in wireless networking and communication in addition to RF monitoring and sensing activities. In this paper, we describe our design, evaluate our solution, present experimental results from multi-sensor deployments and discuss the value of this system in future experimentation.