Barbara Bruno

John Pravin Arockiasamy, Andy Comeca, Victoria Yang et al.

On the way toward full autonomy, sharing roads between automated and autonomous vehicles in so-called mixed traffic is unavoidable. Moreover, even if all vehicles on the road were autonomous, pedestrians would still cross streets. We propose social robots as moderators between autonomous vehicles and vulnerable road users. This paper presents a first proof-of-concept integration of a social robot advising pedestrians in crossing scenarios involving a cooperative automated vehicle. We also discuss key enablers required for designing "robot policeman" in a generic use case of cooperative intersection management. Our work provides a vision of the role of social robotics in future Cooperative Intelligent Transport Systems.

Robin Amann, Zhaolin Li, Barbara Bruno et al.

Speech disfluency commonly occurs in conversational and spontaneous speech. However, standard Automatic Speech Recognition (ASR) models struggle to accurately recognize these disfluencies because they are typically trained on fluent transcripts. Current research mainly focuses on detecting disfluencies within transcripts, overlooking their exact location and duration in the speech. Additionally, previous work often requires model fine-tuning and addresses limited types of disfluencies. In this work, we present an inference-only approach to augment any ASR model with the ability to detect open-set disfluencies. We first demonstrate that ASR models have difficulty transcribing speech disfluencies. Next, this work proposes a modified Connectionist Temporal Classification(CTC)-based forced alignment algorithm from \cite{kurzinger2020ctc} to predict word-level timestamps while effectively capturing disfluent speech. Additionally, we develop a model to classify alignment gaps between timestamps as either containing disfluent speech or silence. This model achieves an accuracy of 81.62% and an F1-score of 80.07%. We test the augmentation pipeline of alignment gap detection and classification on a disfluent dataset. Our results show that we captured 74.13% of the words that were initially missed by the transcription, demonstrating the potential of this pipeline for downstream tasks.

Jérôme Brender, Laila El-Hamamsy, Barbara Bruno et al.

Research has shown that Educational Robotics (ER) enhances student performance, interest, engagement and collaboration. However, until now, the adoption of robotics in formal education has remained relatively scarce. Among other causes, this is due to the difficulty of determining the alignment of educational robotic learning activities with the learning outcomes envisioned by the curriculum, as well as their integration with traditional, non-robotics learning activities that are well established in teachers' practices. This work investigates the integration of ER into formal mathematics education, through a quasi-experimental study employing the Thymio robot and Scratch programming to teach geometry to two classes of 15-year-old students, for a total of 26 participants. Three research questions were addressed: (1) Should an ER-based theoretical lecture precede, succeed or replace a traditional theoretical lecture? (2) What is the students' perception of and engagement in the ER-based lecture and exercises? (3) Do the findings differ according to students' prior appreciation of mathematics? The results suggest that ER activities are as valid as traditional ones in helping students grasp the relevant theoretical concepts. Robotics activities seem particularly beneficial during exercise sessions: students freely chose to do exercises that included the robot, rated them as significantly more interesting and useful than their traditional counterparts, and expressed their interest in introducing ER in other mathematics lectures. Finally, results were generally consistent between the students that like and did not like mathematics, suggesting the use of robotics as a means to broaden the number of students engaged in the discipline.

Morgane Chevalier, Laila El-Hamamsy, Christian Giang et al.

With the introduction of educational robotics (ER) and computational thinking (CT) in classrooms, there is a rising need for operational models that help ensure that CT skills are adequately developed. One such model is the Creative Computational Problem Solving Model (CCPS) which can be employed to improve the design of ER learning activities. Following the first validation with students, the objective of the present study is to validate the model with teachers, specifically considering how they may employ the model in their own practices. The Utility, Usability and Acceptability framework was leveraged for the evaluation through a survey analysis with 334 teachers. Teachers found the CCPS model useful to foster transversal skills but could not recognise the impact of specific intervention methods on CT-related cognitive processes. Similarly, teachers perceived the model to be usable for activity design and intervention, although felt unsure about how to use it to assess student learning and adapt their teaching accordingly. Finally, the teachers accepted the model, as shown by their intent to replicate the activity in their classrooms, but were less willing to modify it or create their own activities, suggesting that they need time to appropriate the model and underlying tenets.

Laila El-Hamamsy, Vaios Papaspyros, Taavet Kangur et al.

Recently, introducing computer science and educational robots in compulsory education has received increasing attention. However, the use of screens in classrooms is often met with resistance, especially in primary school. To address this issue, this study presents the development of a handwriting-based programming language for educational robots. Aiming to align better with existing classroom practices, it allows students to program a robot by drawing symbols with ordinary pens and paper. Regular smartphones are leveraged to process the hand-drawn instructions using computer vision and machine learning algorithms, and send the commands to the robot for execution. To align with the local computer science curriculum, an appropriate playground and scaffolded learning tasks were designed. The system was evaluated in a preliminary test with eight teachers, developers and educational researchers. While the participants pointed out that some technical aspects could be improved, they also acknowledged the potential of the approach to make computer science education in primary school more accessible.

Utku Norman, Tanvi Dinkar, Barbara Bruno et al.

A dialogue is successful when there is alignment between the speakers at different linguistic levels. In this work, we consider the dialogue occurring between interlocutors engaged in a collaborative learning task, where they are not only evaluated on how well they performed, but also on how much they learnt. The main contribution of this work is to propose new automatic measures to study alignment; focusing on verbal (lexical) alignment, and behavioral alignment (when an instruction given by one was followed with concrete actions by another). A second contribution of our work is to study how spontaneous speech phenomena are used in the process of alignment. Lastly, we make public the dataset to study alignment in educational dialogues. Our results show that all teams verbally and behaviourally align to some degree regardless of their performance and learning, and our measures capture that teams that did not succeed in the task were simply slower to collaborate. Thus we find that teams that performed better, were faster to align. Furthermore, our methodology captures a productive period that includes the time where the interlocutors came up with their best solutions. We also find that well-performing teams verbalise the marker "oh" more when they are behaviourally aligned, compared to other times in the dialogue; showing that this marker is an important cue in alignment. To the best of our knowledge, we are the first to study the role of "oh" as an information management marker in a behavioral context (i.e. in connection to actions taken in a physical environment), compared to only a verbal one. Our measures contribute to the research in the field of educational dialogue and the intersection between dialogue and collaborative learning research.

Alessandro Carfi, Carola Motolese, Barbara Bruno et al.

Gestures are a natural communication modality for humans. The ability to interpret gestures is fundamental for robots aiming to naturally interact with humans. Wearable sensors are promising to monitor human activity, in particular the usage of triaxial accelerometers for gesture recognition have been explored. Despite this, the state of the art presents lack of systems for reliable online gesture recognition using accelerometer data. The article proposes SLOTH, an architecture for online gesture recognition, based on a wearable triaxial accelerometer, a Recurrent Neural Network (RNN) probabilistic classifier and a procedure for continuous gesture detection, relying on modelling gesture probabilities, that guarantees (i) good recognition results in terms of precision and recall, (ii) immediate system reactivity.

Barbara Bruno, Fulvio Mastrogiovanni, Federico Pecora et al.

Cultural adaptation, i.e., the matching of a robot's behaviours to the cultural norms and preferences of its user, is a well known key requirement for the success of any assistive application. However, culture-dependent robot behaviours are often implicitly set by designers, thus not allowing for an easy and automatic adaptation to different cultures. This paper presents a method for the design of culture-aware robots, that can automatically adapt their behaviour to conform to a given culture. We propose a mapping from cultural factors to related parameters of robot behaviours which relies on linguistic variables to encode heterogeneous cultural factors in a uniform formalism, and on fuzzy rules to encode qualitative relations among multiple variables. We illustrate the approach in two practical case studies.

Barbara Bruno, Nak Young Chong, Hiroko Kamide et al.

Cultural competence is a well known requirement for an effective healthcare, widely investigated in the nursing literature. We claim that personal assistive robots should likewise be culturally competent, aware of general cultural characteristics and of the different forms they take in different individuals, and sensitive to cultural differences while perceiving, reasoning, and acting. Drawing inspiration from existing guidelines for culturally competent healthcare and the state-of-the-art in culturally competent robotics, we identify the key robot capabilities which enable culturally competent behaviours and discuss methodologies for their development and evaluation.

Roberto Menicatti, Barbara Bruno, Antonio Sgorbissa

Daily life activities, such as eating and sleeping, are deeply influenced by a person's culture, hence generating differences in the way a same activity is performed by individuals belonging to different cultures. We argue that taking cultural information into account can improve the performance of systems for the automated recognition of human activities. We propose four different solutions to the problem and present a system which uses a Naive Bayes model to associate cultural information with semantic information extracted from still images. Preliminary experiments with a dataset of images of individuals lying on the floor, sleeping on a futon and sleeping on a bed suggest that: i) solutions explicitly taking cultural information into account are more accurate than culture-unaware solutions; and ii) the proposed system is a promising starting point for the development of culture-aware Human Activity Recognition methods.

Luca Buoncompagni, Barbara Bruno, Antonella Giuni et al.

Providing elderly and people with special needs, including those suffering from physical disabilities and chronic diseases, with the possibility of retaining their independence at best is one of the most important challenges our society is expected to face. Assistance models based on the home care paradigm are being adopted rapidly in almost all industrialized and emerging countries. Such paradigms hypothesize that it is necessary to ensure that the so-called Activities of Daily Living are correctly and regularly performed by the assisted person to increase the perception of an improved quality of life. This chapter describes the computational inference engine at the core of Arianna, a system able to understand whether an assisted person performs a given set of ADL and to motivate him/her in performing them through a speech-mediated motivational dialogue, using a set of nearables to be installed in an apartment, plus a wearable to be worn or fit in garments.

Barbara Bruno, Nak Young Chong, Hiroko Kamide et al.

The nursing literature shows that cultural competence is an important requirement for effective healthcare. We claim that personal assistive robots should likewise be culturally competent, that is, they should be aware of general cultural characteristics and of the different forms they take in different individuals, and take these into account while perceiving, reasoning, and acting. The CARESSES project is an Europe-Japan collaborative effort that aims at designing, developing and evaluating culturally competent assistive robots. These robots will be able to adapt the way they behave, speak and interact to the cultural identity of the person they assist. This paper describes the approach taken in the CARESSES project, its initial steps, and its future plans.

Luca Buoncompagni, Barbara Bruno, Antonella Giuni et al.

Providing elderly and people with special needs to retain their independence as long as possible is one of the biggest challenges of the society of tomorrow. Teseo, a startup company spinoff from the University of Genoa, aims at accelerating the transition towards a sustainable healthcare system. Teseo's first concept and product, Arianna, allows for the automated recognition of activities of daily living at home and acts as a wellbeing and healthcare personalized assistant. This abstract outlines the main concepts underlying its features and capabilities.

Divya Shah, Ernesto Denicia, Tiago Pimentel et al.

Bimanual gestures are of the utmost importance for the study of motor coordination in humans and in everyday activities. A reliable detection of bimanual gestures in unconstrained environments is fundamental for their clinical study and to assess common activities of daily living. This paper investigates techniques for a reliable, unconstrained detection and classification of bimanual gestures. It assumes the availability of inertial data originating from the two hands/arms, builds upon a previously developed technique for gesture modelling based on Gaussian Mixture Modelling (GMM) and Gaussian Mixture Regression (GMR), and compares different modelling and classification techniques, which are based on a number of assumptions inspired by literature about how bimanual gestures are represented and modelled in the brain. Experiments show results related to 5 everyday bimanual activities, which have been selected on the basis of three main parameters: (not) constraining the two hands by a physical tool, (not) requiring a specific sequence of single-hand gestures, being recursive (or not). In the best performing combination of modeling approach and classification technique, five out of five activities are recognized up to an accuracy of 97%, a precision of 82% and a level of recall of 100%.

Kourosh Darwish, Francesco Wanderlingh, Barbara Bruno et al.

The Industry 4.0 paradigm emphasizes the crucial benefits that collaborative robots, i.e., robots able to work alongside and together with humans, could bring to the whole production process. In this context, an enabling technology yet unreached is the design of flexible robots able to deal at all levels with humans' intrinsic variability, which is not only a necessary element for a comfortable working experience for the person but also a precious capability for efficiently dealing with unexpected events. In this paper, a sensing, representation, planning and control architecture for flexible human-robot cooperation, referred to as FlexHRC, is proposed. FlexHRC relies on wearable sensors for human action recognition, AND/OR graphs for the representation of and reasoning upon cooperation models, and a Task Priority framework to decouple action planning from robot motion planning and control.