Hailong Liu

Hailong Liu, Junya Wada, Toshihiro Hiraoka et al.

Drivers with peripheral visual field defects may fail to notice pedestrians in their peripheral visual field, leading to delayed hazard awareness and increased collision risk. This study explores hanger reflex cue (HRC) as a driving assistance method for drivers with peripheral visual field defects, in which mechanical pressure is applied to specific regions of the head to facilitate anticipatory orientation toward potentially risky pedestrians and support safer driving. In a driving simulator experiment with 15 participants, we compared driving behavior with and without HRC during pedestrian encounters under simulated peripheral visual field defect. The results showed that HRC significantly shifted drivers' modal head rotation angle toward the risky pedestrian and significantly increased gaze duration toward that pedestrian. Collision occurrence was lower in the w/ HRC condition than in the w/o HRC condition, although the direct effect of HRC on collision occurrence showed only a marginal trend. A piecewise structural equation modeling analysis further suggested that HRC may contribute to collision reduction through a sequential pathway from head rotation to gaze allocation and then to collision occurrence. These findings provide preliminary evidence that HRC can support anticipatory attention allocation toward peripheral hazards and may offer a promising driving assistance method for drivers with visual field impairment.

Zhe Zeng, Clara Ayumi Fechner, Fei Yan et al.

With the increasing deployment of robots in public spaces, encounters between robots and incidentally copresent persons (InCoPs) are becoming more frequent. However, InCoPs remain largely underexplored in the literature, particularly from a cross-cultural perspective. Therefore, the present study investigates cultural differences in InCoPs' existence acceptance (EA) of autonomous cleaning robots in public spaces among Japanese and German participants. Online survey results revealed that Germans showed significantly higher EA. Social Norms and Trust were the strongest positive EA predictors across cultures. More specifically, for Germans, EA was directly influenced by Usefulness, Interest and Anger, showing a functional-affective pattern where functional perceptions boost EA and anger suppresses it. For Japanese participants, Trust, Surprise and Fear were the direct associational factors, forming a trust-emotion pattern. These findings reveal cultural influences on cognitive and emotional drivers of public robot acceptance, emphasizing the need for culturally adaptive robot design.

Hailong Liu, Masaki Kuge, Toshihiro Hiraoka et al.

Level 3 automated vehicles (AVs) issue a request to intervene (RtI) when the automated driving system approaches its system limitations. Although this takeover transition is safety-critical, it is usually invisible to surrounding manually driven vehicle (MV) drivers. This study proposes an external human-machine interface (eHMI) called eHMI C+O that externalizes the RtI-related takeover status of a Level~3 AV using cyan and orange light bars. A driving-simulator experiment with 40 participants examined whether the proposed eHMI supports surrounding MV drivers during AV takeover scenarios. The results showed that, compared with the ADS-status-only eHMI condition, which is similar to ``Automated Driving Marker Lights,'' and the no-eHMI condition, the proposed eHMI C+O significantly improved participants' understanding of the AV's driving intention, their prediction of its behavior, and their perceived sufficiency of the information presented by the AV. It also reduced hesitation, increased confidence, and promoted earlier and larger increases in time headway after the RtI was issued. In the AV accident scenario, eHMI C+O significantly reduced the odds of accident involvement for the following MV compared with the no-eHMI condition, corresponding to a 76.8% reduction in accident odds. Exploratory path analysis suggested that the safety benefit of the proposed eHMI C+O may be associated with improved situation awareness and earlier defensive driving responses. These findings indicate that externalizing RtI-related takeover status can help surrounding drivers better understand Level 3 AVs and respond more safely during safety-critical takeover transitions.

Yuya Ide, Hailong Liu, Takahiro Wada

Autonomous personal mobility vehicles (APMVs) are novel smart mobility devices designed to provide automated individual transportation in indoor or mixed-traffic environments. However, in such environments, frequent pedestrian avoidance maneuvers may cause rapid steering adjustments and passive postural responses from passengers, thereby increasing the risk of motion sickness. This study investigated whether indicating the future driving path could mitigate motion sickness in APMV passengers. A mixed-design experiment was conducted with 40 participants under two self-reported genders as a between-subject factor (male and female), two driving paths as a between-subject factor (irregular and regular) and three driving conditions as a within-subject factor (manual driving (MD), automated driving without path indication (AD w/o path), and automated driving with path indication (AD w/ path)). Motion sickness was evaluated using the Motion Illness Symptom Classification (MISC), and head motion was assessed by calculating the delay time of participants' head yaw rate relative to APMV's yaw rate in the turning direction. The results showed that driving condition was the only factor that significantly affected both motion sickness and head-motion delay. Compared with the AD w/o path condition, both the MD and AD w/ path conditions were associated with lower motion sickness severity, longer motion sickness onset latency, and earlier head motion relative to vehicle motion. Notably, the AD w/ path condition achieved motion sickness levels comparable to those in the MD condition. Furthermore, repeated-measures correlation analysis showed significant associations between head-motion delay and all MISC metrics but the underlying physiological mechanism remains to be elucidated. These findings suggest that presenting information about future driving path can mitigate motion sickness in APMV passengers.

Ryuji Matsuo, Hailong Liu, Toshihiro Hiraoka et al.

Level 3 automated driving systems (ADS) have attracted significant attention and are being commercialized. A level 3 ADS prompts the driver to take control by issuing a request to intervene (RtI) when its operational design domains (ODD) are exceeded. However, complex traffic situations can cause drivers to perceive multiple potential triggers of RtI simultaneously, causing hesitation or confusion during take-over. Therefore, drivers need to clearly understand the ADS's system limitations to ensure safe take-over. This study proposes a voice-based educational human machine interface~(HMI) for providing RtI trigger cues and reason to help drivers understand ADS's system limitations. The results of a between-group experiment using a driving simulator showed that incorporating effective trigger cues and reason into the RtI was related to improved driver comprehension of the ADS's system limitations. Moreover, most participants, instructed via the proposed method, could proactively take over control of the ADS in cases where RtI fails; meanwhile, their number of collisions was lower compared with the other RtI HMI conditions. Therefore, using the proposed method to continually enhance the driver's understanding of the system limitations of ADS through the proposed method is associated with safer and more effective real-time interactions with ADS.

Shengfu Chen, Hailong Liu, Wenzhao Wei

This report presents the approach adopted in the Modelscope-Sora challenge, which focuses on fine-tuning data for video generation models. The challenge evaluates participants' ability to analyze, clean, and generate high-quality datasets for video-based text-to-video tasks under specific computational constraints. The provided methodology involves data processing techniques such as video description generation, filtering, and acceleration. This report outlines the procedures and tools utilized to enhance the quality of training data, ensuring improved performance in text-to-video generation models.

Hailong Liu, Shota Inoue, Takahiro Wada

Passengers (drivers) of level 3-5 autonomous personal mobility vehicles (APMV) and cars can perform non-driving tasks, such as reading books and smartphones, while driving. It has been pointed out that such activities may increase motion sickness. Many studies have been conducted to build countermeasures, of which various computational motion sickness models have been developed. Many of these are based on subjective vertical conflict (SVC) theory, which describes vertical changes in direction sensed by human sensory organs vs. those expected by the central nervous system. Such models are expected to be applied to autonomous driving scenarios. However, no current computational model can integrate visual vertical information with vestibular sensations. We proposed a 6 DoF SVC-VV model which add a visually perceived vertical block into a conventional six-degrees-of-freedom SVC model to predict VV directions from image data simulating the visual input of a human. Hence, a simple image-based VV estimation method is proposed. As the validation of the proposed model, this paper focuses on describing the fact that the motion sickness increases as a passenger reads a book while using an AMPV, assuming that visual vertical (VV) plays an important role. In the static experiment, it is demonstrated that the estimated VV by the proposed method accurately described the gravitational acceleration direction with a low mean absolute deviation. In addition, the results of the driving experiment using an APMV demonstrated that the proposed 6 DoF SVC-VV model could describe that the increased motion sickness experienced when the VV and gravitational acceleration directions were different.

Zhe Zeng, Sai Liu, Hao Cheng et al.

Even before the Covid-19 pandemic, beneficial use cases for hygienic, touchless human-machine interaction have been explored. Gaze input, i.e., information input via eye-movements of users, represents a promising method for contact-free interaction in human-machine systems. In this paper, we present the GazeVending interface (GaVe), which lets users control actions on a display with their eyes. The interface works on a regular webcam, available on most of today's laptops, and only requires a one-point calibration before use. GaVe is designed in a hierarchical structure, presenting broad item cluster to users first and subsequently guiding them through another selection round, which allows the presentation of a large number of items. Cluster/item selection in GaVe is based on the dwell time of fixations, i.e., the time duration that users look at a given Cluster/item. A user study (N=22) was conducted to test optimal dwell time thresholds and comfortable human-to-display distances. Users' perception of the system, as well as error rates and task completion time were registered. We found that all participants were able to use the system with a short time training, and showed good performance during system usage, selecting a target item within a group of 12 items in 6.76 seconds on average. Participants were able to quickly understand and know how to interact with the interface. We provide design guidelines for GaVe and discuss the potentials of the system.

Hao Cheng, Li Feng, Hailong Liu et al.

Intersections where vehicles are permitted to turn and interact with vulnerable road users (VRUs) like pedestrians and cyclists are among some of the most challenging locations for automated and accurate recognition of road users' behavior. In this paper, we propose a deep conditional generative model for interaction detection at such locations. It aims to automatically analyze massive video data about the continuity of road users' behavior. This task is essential for many intelligent transportation systems such as traffic safety control and self-driving cars that depend on the understanding of road users' locomotion. A Conditional Variational Auto-Encoder based model with Gaussian latent variables is trained to encode road users' behavior and perform probabilistic and diverse predictions of interactions. The model takes as input the information of road users' type, position and motion automatically extracted by a deep learning object detector and optical flow from videos, and generates frame-wise probabilities that represent the dynamics of interactions between a turning vehicle and any VRUs involved. The model's efficacy was validated by testing on real--world datasets acquired from two different intersections. It achieved an F1-score above 0.96 at a right--turn intersection in Germany and 0.89 at a left--turn intersection in Japan, both with very busy traffic flows.

Yang Li, Hao Cheng, Zhe Zeng et al.

In comparison to conventional traffic designs, shared spaces promote a more pleasant urban environment with slower motorized movement, smoother traffic, and less congestion. In the foreseeable future, shared spaces will be populated with a mixture of autonomous vehicles (AVs) and vulnerable road users (VRUs) like pedestrians and cyclists. However, a driver-less AV lacks a way to communicate with the VRUs when they have to reach an agreement of a negotiation, which brings new challenges to the safety and smoothness of the traffic. To find a feasible solution to integrating AVs seamlessly into shared-space traffic, we first identified the possible issues that the shared-space designs have not considered for the role of AVs. Then an online questionnaire was used to ask participants about how they would like a driver of the manually driving vehicle to communicate with VRUs in a shared space. We found that when the driver wanted to give some suggestions to the VRUs in a negotiation, participants thought that the communications via the driver's body behaviors were necessary. Besides, when the driver conveyed information about her/his intentions and cautions to the VRUs, participants selected different communication methods with respect to their transport modes (as a driver, pedestrian, or cyclist). These results suggest that novel eHMIs might be useful for AV-VRU communication when the original drivers are not present. Hence, a potential eHMI design concept was proposed for different VRUs to meet their various expectations. In the end, we further discussed the effects of the eHMIs on improving the sociality in shared spaces and the autonomous driving systems.

Hailong Liu, Takatsugu Hirayama, Masaya Watanabe

Compared to a manual driving vehicle (MV), an automated driving vehicle lacks a way to communicate with the pedestrian through the driver when it interacts with the pedestrian because the driver usually does not participate in driving tasks. Thus, an external human machine interface (eHMI) can be viewed as a novel explicit communication method for providing driving intentions of an automated driving vehicle (AV) to pedestrians when they need to negotiate in an interaction, e.g., an encountering scene. However, the eHMI may not guarantee that the pedestrians will fully recognize the intention of the AV. In this paper, we propose that the instruction of the eHMI's rationale can help pedestrians correctly understand the driving intentions and predict the behavior of the AV, and thus their subjective feelings (i.e., dangerous feeling, trust in the AV, and feeling of relief) and decision-making are also improved. The results of an interaction experiment in a road-crossing scene indicate that the participants were more difficult to be aware of the situation when they encountered an AV w/o eHMI compared to when they encountered an MV; further, the participants' subjective feelings and hesitation in decision-making also deteriorated significantly. When the eHMI was used in the AV, the situational awareness, subjective feelings and decision-making of the participants regarding the AV w/ eHMI were improved. After the instruction, it was easier for the participants to understand the driving intention and predict driving behavior of the AV w/ eHMI. Further, the subjective feelings and the hesitation related to decision-making were improved and reached the same standards as that for the MV.

Hailong Liu, Takatsugu Hirayama, Luis Yoichi Morales et al.

Although automated driving systems have been used frequently, they are still unpopular in society. To increase the popularity of automated vehicles (AVs), assisting pedestrians to accurately understand the driving intentions and improving their perception of safety when interacting with AVs are considered effective. Therefore, the AV should send information about its driving intention to pedestrians when they interact with each other. However, the following questions should be answered regarding how the AV sends the information to them: 1) What timing for an AV to make pedestrians understand its driving intentions after being noticed by them? 2) What timing for an AV to make pedestrians feel safe after being noticed by them? Thirteen participants were invited to interact with a manually driven vehicle and an AV in an experiment. The participants' gaze information and a subjective evaluation of their understanding of the driving intention as well as their perception of safety were collected. By analyzing the participants' gaze duration on the vehicle with their subjective evaluations, we found that the AV should enable the pedestrian to accurately understand its driving intention within 0.5~6.5 [s] and make the pedestrian feel safe within 0.5~8.0 [s] while the pedestrian is gazing at it.

Hailong Liu, Takatsugu Hirayama, Luis Yoichi Morales et al.

Interactions between pedestrians and automated vehicles (AVs) will increase significantly with the popularity of AV. However, pedestrians often have not enough trust on the AVs , particularly when they are confused about an AV's intention in a interaction. This study seeks to evaluate if pedestrians clearly understand the driving intentions of AVs in interactions and presents experimental research on the relationship between gaze behaviors of pedestrians and their understanding of the intentions of the AV. The hypothesis investigated in this study was that the less the pedestrian understands the driving intentions of the AV, the longer the duration of their gazing behavior will be. A pedestrian--vehicle interaction experiment was designed to verify the proposed hypothesis. A robotic wheelchair was used as the manual driving vehicle (MV) and AV for interacting with pedestrians while pedestrians' gaze data and their subjective evaluation of the driving intentions were recorded. The experimental results supported our hypothesis as there was a negative correlation between the pedestrians' gaze duration on the AV and their understanding of the driving intentions of the AV. Moreover, the gaze duration of most of the pedestrians on the MV was shorter than that on an AV. Therefore, we conclude with two recommendations to designers of external human-machine interfaces (eHMI): (1) when a pedestrian is engaged in an interaction with an AV, the driving intentions of the AV should be provided; (2) if the pedestrian still gazes at the AV after the AV displays its driving intentions, the AV should provide clearer information about its driving intentions.

Hailong Liu, Toshihiro Hiraoka, Seiya Tanaka

As various driving automation system (DAS) are commonly used in the vehicle, the over-trust in the DAS may put the driver in the risk. In order to prevent the over-trust while driving, the trust state of the driver should be recognized. However, description variables of the trust state are not distinct. This paper assumed that the outward expressions of a driver can represent the trust state of him/her-self. The explicit behaviors when driving with DAS is seen as those outward expressions. In the experiment, a driving simulator with a driver monitoring system was used for simulating a vehicle with the adaptive cruise control (ACC) and observing the motion information of the driver. Results show that if the driver completely trusted in the ACC, then 1) the participants were likely to put their feet far away from the pedals; 2) the operational intervention of the driver will delay in dangerous situations. In the future, a machine learning model will be tried to predict the trust state by using the motion information of the driver.

Hailong Liu, Toshihiro Hiraoka, Takatsugu Hirayama et al.

The head-up display (HUD) is an emerging device which can project information on a transparent screen. The HUD has been used in airplanes and vehicles, and it is usually placed in front of the operator's view. In the case of the vehicle, the driver can see not only various information on the HUD but also the backgrounds (driving environment) through the HUD. However, the projected information on the HUD may interfere with the colors in the background because the HUD is transparent. For example, a red message on the HUD will be less noticeable when there is an overlap between it and the red brake light from the front vehicle. As the first step to solve this issue, how to evaluate the mutual interference between the information on the HUD and backgrounds is important. Therefore, this paper proposes a method to evaluate the mutual interference based on saliency. It can be evaluated by comparing the HUD part cut from a saliency map of a measured image with the HUD image.

Hailong Liu, Toshihiro Hiraoka

Levels one to three of driving automation systems~(DAS) are spreading fast. However, as the DAS functions become more and more sophisticated, not only the driver's driving skills will reduce, but also the problem of over-trust will become serious. If a driver has over-trust in the DAS, he/she will become not aware of hazards in time. To prevent the driver's over-trust in the DAS, this paper discusses the followings: 1) the definition of over-trust in the DAS, 2) a hypothesis of occurrence condition and occurrence process of over-trust in the DAS, and 3) a driving behavior model based on the trust in the DAS, the risk homeostasis theory, and the over-trust prevention human-machine interface.