Panagiotis Kourtesis

Panagiotis Kourtesis, Sarah E. MacPherson

Clinical tools involving immersive virtual reality (VR) may bring several advantages to cognitive neuroscience and neuropsychology. However, there are some technical and methodological pitfalls. The American Academy of Clinical Neuropsychology (AACN) and the National Academy of Neuropsychology (NAN) raised 8 key issues pertaining to Computerized Neuropsychological Assessment Devices. These issues pertain to: (1) the safety and effectivity; (2) the identity of the end-user; (3) the technical hardware and software features; (4) privacy and data security; (5) the psychometric properties; (6) examinee issues; (7) the use of reporting services; and (8) the reliability of the responses and results. The VR Everyday Assessment Lab (VR-EAL) is the first immersive VR neuropsychological battery with enhanced ecological validity for the assessment of everyday cognitive functions by offering a pleasant testing experience without inducing cybersickness. The VR-EAL meets the criteria of the NAN and AACN, addresses the methodological pitfalls, and brings advantages for neuropsychological testing. However, there are still shortcomings of the VR-EAL, which should be addressed. Future iterations should strive to improve the embodiment illusion in VR-EAL and the creation of an open access VR software library should be attempted. The discussed studies demonstrate the utility of VR methods in cognitive neuroscience and neuropsychology.

Panagiotis Kourtesis, Ferran Argelaguet, Sebastian Vizcay et al.

Haptic feedback is critical in a broad range of human-machine/computer-interaction applications. However, the high cost and low portability/wearability of haptic devices remain unresolved issues, severely limiting the adoption of this otherwise promising technology. Electrotactile interfaces have the advantage of being more portable and wearable due to their reduced actuators' size, as well as their lower power consumption and manufacturing cost. The applications of electrotactile feedback have been explored in human-computer interaction and human-machine-interaction for facilitating hand-based interactions in applications such as prosthetics, virtual reality, robotic teleoperation, surface haptics, portable devices, and rehabilitation. This paper presents a technological overview of electrotactile feedback, as well a systematic review and meta-analysis of its applications for hand-based interactions. We discuss the different electrotactile systems according to the type of application. We also discuss over a quantitative congregation of the findings, to offer a high-level overview into the state-of-art and suggest future directions. Electrotactile feedback systems showed increased portability/wearability, and they were successful in rendering and/or augmenting most tactile sensations, eliciting perceptual processes, and improving performance in many scenarios. However, knowledge gaps (e.g., embodiment), technical (e.g., recurrent calibration, electrodes' durability) and methodological (e.g., sample size) drawbacks were detected, which should be addressed in future studies.

Panagiotis Kourtesis, Sarah E. MacPherson

Studies on prospective memory (PM) predominantly assess either event- or time-based PM by implementing non-ecological laboratory-based tasks. The results deriving from these paradigms have provided findings that are discrepant with ecologically valid research paradigms that converge on the complexity and cognitive demands of everyday tasks. The Virtual Reality Everyday Assessment Lab (VR-EAL), an immersive virtual reality (VR) neuropsychological battery with enhanced ecological validity, was implemented to assess everyday event- and time-based PM, as well as the influence of other cognitive functions on everyday PM functioning. The results demonstrated the importance of delayed recognition, planning, and visuospatial attention on everyday PM. Delayed recognition and planning ability were found to be central in event- and time-based PM respectively. In order of importance, delayed recognition, visuospatial attention speed, and planning ability were found to be involved in event-based PM functioning. Comparably, planning, visuospatial attention accuracy, delayed recognition, and multitasking/task-shifting ability were found to be involved in time-based PM functioning. These findings further suggest the importance of ecological validity in the study of PM, which may be achieved using immersive VR paradigms.

Panagiotis Kourtesis, Simona Collina, Leonidas A. A. Doumas et al.

Recent research has focused on assessing either event- or time-based prospective memory (PM) using laboratory tasks. Yet, the findings pertaining to PM performance on laboratory tasks are often inconsistent with the findings on corresponding naturalistic experiments. Ecologically valid neuropsychological tasks resemble the complexity and cognitive demands of everyday tasks, offer an adequate level of experimental control, and allow a generalisation of the findings to everyday performance. The Virtual Reality Everyday Assessment Lab (VR-EAL), an immersive virtual reality neuropsychological battery with enhanced ecological validity, was implemented to comprehensively assess everyday PM (i.e., focal and non-focal event-based, and time-based). The effects of the length of delay between encoding and initiating the PM intention and the type of PM task on everyday PM performance were examined. The results revealed that everyday PM performance was affected by the length of delay rather than the type of PM task. The effect of the length of delay differentially affected performance on the focal, non-focal, and time-based tasks and was proportional to the PM cue focality (i.e., semantic relationship with the intended action). This study also highlighted methodological considerations such as the differentiation between functioning and ability, distinction of cue attributes, and the necessity of ecological validity.

Sebastian Vizcay, Panagiotis Kourtesis, Ferran Argelaguet et al.

This paper presents a wearable electrotactile feedback system to enhance contact information for mid-air interactions with virtual objects. In particular, we propose the use of electrotactile feedback to render the interpenetration distance between the user's finger and the virtual content is touched. Our approach consists of modulating the perceived intensity (frequency and pulse width modulation) of the electrotactile stimuli according to the registered interpenetration distance. In a user study (N=21), we assessed the performance of four different interpenetration feedback approaches: electrotactile-only, visual-only, electrotactile and visual, and no interpenetration feedback. First, the results showed that contact precision and accuracy were significantly improved when using interpenetration feedback. Second, and more interestingly, there were no significant differences between visual and electrotactile feedback when the calibration was optimized and the user was familiarized with electrotactile feedback. Taken together, these results suggest that electrotactile feedback could be an efficient replacement of visual feedback for enhancing contact information in virtual reality avoiding the need of active visual focus and the rendering of additional visual artefacts.

Panagiotis Kourtesis, Danai Korre, Simona Collina et al.

Virtual reality (VR) head-mounted displays (HMD) appear to be effective research tools, which may address the problem of ecological validity in neuropsychological testing. However, their widespread implementation is hindered by VR induced symptoms and effects (VRISE) and the lack of skills in VR software development. This study offers guidelines for the development of VR software in cognitive neuroscience and neuropsychology, by describing and discussing the stages of the development of Virtual Reality Everyday Assessment Lab (VR-EAL), the first neuropsychological battery in immersive VR. Techniques for evaluating cognitive functions within a realistic storyline are discussed. The utility of various assets in Unity, software development kits, and other software are described so that cognitive scientists can overcome challenges pertinent to VRISE and the quality of the VR software. In addition, this pilot study attempts to evaluate VR-EAL in accordance with the necessary criteria for VR software for research purposes. The VR neuroscience questionnaire (VRNQ; Kourtesis et al., 2019b) was implemented to appraise the quality of the three versions of VR-EAL in terms of user experience, game mechanics, in-game assistance, and VRISE. Twenty-five participants aged between 20 and 45 years with 12-16 years of full-time education evaluated various versions of VR-EAL. The final version of VR-EAL achieved high scores in every sub-score of the VRNQ and exceeded its parsimonious cut-offs. It also appeared to have better in-game assistance and game mechanics, while its improved graphics substantially increased the quality of the user experience and almost eradicated VRISE. The results substantially support the feasibility of the development of effective VR research and clinical software without the presence of VRISE during a 60-minute VR session.

Panagiotis Kourtesis, Simona Collina, Leonidas A. A. Doumas et al.

Research suggests that the duration of a VR session modulates the presence and intensity of VRISE, but there are no suggestions regarding the appropriate maximum duration of VR sessions. The implementation of high-end VR HMDs in conjunction with ergonomic VR software seems to mitigate the presence of VRISE substantially. However, a brief tool does not currently exist to appraise and report both the quality of software features and VRISE intensity quantitatively. The VRNQ was developed to assess the quality of VR software in terms of user experience, game mechanics, in-game assistance, and VRISE. Forty participants aged between 28 and 43 years were recruited (18 gamers and 22 non-gamers) for the study. They participated in 3 different VR sessions until they felt weary or discomfort and subsequently filled in the VRNQ. Our results demonstrated that VRNQ is a valid tool for assessing VR software as it has good convergent, discriminant, and construct validity. The maximum duration of VR sessions should be between 55-70 minutes when the VR software meets or exceeds the parsimonious cut-offs of the VRNQ and the users are familiarized with the VR system. Also. the gaming experience does not seem to affect how long VR sessions should last. Also, while the quality of VR software substantially modulates the maximum duration of VR sessions, age and education do not. Finally, deeper immersion, better quality of graphics and sound, and more helpful in-game instructions and prompts were found to reduce VRISE intensity. The VRNQ facilitates the brief assessment and reporting of the quality of VR software features and/or the intensity of VRISE, while its minimum and parsimonious cut-offs may appraise the suitability of VR software. The findings of this study contribute to the establishment of rigorous VR methods that are crucial for the viability of immersive VR as a research and clinical tool.

Panagiotis Kourtesis, Simona Collina, Leonidas A. A. Doumas et al.

Immersive virtual reality (VR) emerges as a promising research and clinical tool. However, several studies suggest that VR induced adverse symptoms and effects (VRISE) may undermine the health and safety standards, and the reliability of the scientific results. In the current literature review, the technical reasons for the adverse symptomatology are investigated to provide suggestions and technological knowledge for the implementation of VR head-mounted display (HMD) systems in cognitive neuroscience. The technological systematic literature indicated features pertinent to display, sound, motion tracking, navigation, ergonomic interactions, user experience, and computer hardware that should be considered by the researchers. Subsequently, a meta-analysis of 44 neuroscientific or neuropsychological studies involving VR HMD systems was performed. The meta-analysis of the VR studies demonstrated that new generation HMDs induced significantly less VRISE and marginally fewer dropouts.Importantly, the commercial versions of the new generation HMDs with ergonomic interactions had zero incidents of adverse symptomatology and dropouts. HMDs equivalent to or greater than the commercial versions of contemporary HMDs accompanied with ergonomic interactions are suitable for implementation in cognitive neuroscience. In conclusion, researchers technological competency, along with meticulous methods and reports pertinent to software, hardware, and VRISE, are paramount to ensure the health and safety standards and the reliability of neuroscientific results.