André Santos

André Santos, Nuno Ferreira Duarte, Atabak Dehban et al.

We tackle the challenge of robotic bin packing with irregular objects, such as groceries. Given the diverse physical attributes of these objects and the complex constraints governing their placement and manipulation, employing preprogrammed strategies becomes unfeasible. Our approach is to learn directly from expert demonstrations in order to extract implicit task knowledge and strategies to ensure safe object positioning, efficient use of space, and the generation of human-like behaviors that enhance human-robot trust. We rely on human demonstrations to learn a Markov chain for predicting the object packing sequence for a given set of items and then compare it with human performance. Our experimental results show that the model outperforms human performance by generating sequence predictions that humans classify as human-like more frequently than human-generated sequences. The human demonstrations were collected using our proposed VR platform, BoxED, which is a box packaging environment for simulating real-world objects and scenarios for fast and streamlined data collection with the purpose of teaching robots. We collected data from 43 participants packing a total of 263 boxes with supermarket-like objects, yielding 4644 object manipulations. Our VR platform can be easily adapted to new scenarios and objects, and is publicly available, alongside our dataset, at https://github.com/andrejfsantos4/BoxED.

André Rodrigues, André Santos, Kyle Montague et al.

Touch data, and in particular text-entry data, has been mostly collected in the laboratory, under controlled conditions. While touch and text-entry data have consistently shown its potential for monitoring and detecting a variety of conditions and impairments, its deployment in-the-wild remains a challenge. In this paper, we present WildKey, an Android keyboard toolkit that allows for the usable deployment of in-the-wild user studies. WildKey is able to analyze text-entry behaviors through implicit and explicit text-entry data collection while ensuring user privacy. We detail each of the WildKey's components and features, all of the metrics collected, and discuss the steps taken to ensure user privacy and promote compliance.

André Rodrigues, André Santos, Kyle Montague et al.

Accessibility assessments typically focus on determining a binary measurement of task performance success/failure; and often neglect to acknowledge the nuances of those interactions. Although a large population of blind people find smartphone interactions possible, many experiences take a significant toll and can have a lasting negative impact on the individual and their willingness to step out of technological comfort zones. There is a need to assist and support individuals with the adoption and learning process of new tasks to mitigate these negative experiences. We contribute with a human-powered nonvisual task assistant for smartphones to provide pervasive assistance. We argue, in addition to success, one must carefully consider promoting and evaluating factors such as self-efficacy and the belief in one's own abilities to control and learn to use technology. In this paper, we show effective assistant positively affects self-efficacy when performing new tasks with smartphones, affects perceptions of accessibility and enables systemic task-based learning.