David Carter

Kirthevasan Kandasamy, Yoram Bachrach, Ryota Tomioka et al.

We study reinforcement learning of chatbots with recurrent neural network architectures when the rewards are noisy and expensive to obtain. For instance, a chatbot used in automated customer service support can be scored by quality assurance agents, but this process can be expensive, time consuming and noisy. Previous reinforcement learning work for natural language processing uses on-policy updates and/or is designed for on-line learning settings. We demonstrate empirically that such strategies are not appropriate for this setting and develop an off-policy batch policy gradient method (BPG). We demonstrate the efficacy of our method via a series of synthetic experiments and an Amazon Mechanical Turk experiment on a restaurant recommendations dataset.

David Carter, Luiz Fernando Capretz

Three-dimension will be a characteristic of future user interfaces, although we are just starting to gain an understanding of how users can navigate and share information within a virtual 3D environment. Three-dimensional graphical user interfaces (3D-GUI) raise many issues of design, metaphor and usability. This research is devoted to designing a 3D-GUI as a front-end tool for a file management system, in this case, for Microsoft Windows\c{opyright} Explorer; as well as evaluating the efficiency of a 3D application. The software design was implemented by extending the Half-Life 3D engine. This extension provides a directory traversal and basic file management functions, like cut, copy, paste, delete, and so on. This paper shows the design and implementation of a real-world application that contains an efficient 3D-GUI.