David R. Glowacki

Mohamed Dhouioui, Jonathan Barnoud, Rhoslyn Roebuck Williams et al.

Molecular dynamics (MD) simulations are a crucial computational tool for researchers to understand and engineer molecular structure and function in areas such as drug discovery, protein engineering, and material design. Despite their utility, MD simulations are expensive, owing to the high dimensionality of molecular systems. Interactive molecular dynamics in virtual reality (iMD-VR) has recently emerged as a "human-in-the-loop" strategy for efficiently navigating hyper-dimensional molecular systems. By providing an immersive 3D environment that enables visualization and manipulation of real-time molecular simulations running on high-performance computing architectures, iMD-VR enables researchers to reach out and guide molecular conformational dynamics, in order to efficiently explore complex, high-dimensional molecular systems. Moreover, iMD-VR simulations generate rich datasets that capture human experts' spatial insight regarding molecular structure and function. This paper explores the use of researcher-generated iMD-VR datasets to train AI agents via imitation learning (IL). IL enables agents to mimic complex behaviours from expert demonstrations, circumventing the need for explicit programming or intricate reward design. In this article, we review IL across robotics and Multi-agents systems domains which are comparable to iMD-VR, and discuss how iMD-VR recordings could be used to train IL models to interact with MD simulations. We then illustrate the applications of these ideas through a proof-of-principle study where iMD-VR data was used to train a CNN network on a simple molecular manipulation task; namely, threading a small molecule through a nanotube pore. Finally, we outline future research directions and potential challenges of using AI agents to augment human expertise in navigating vast molecular conformational spaces.

Lars A. Bratholm, Will Gerrard, Brandon Anderson et al.

The rise of machine learning (ML) has created an explosion in the potential strategies for using data to make scientific predictions. For physical scientists wishing to apply ML strategies to a particular domain, it can be difficult to assess in advance what strategy to adopt within a vast space of possibilities. Here we outline the results of an online community-powered effort to swarm search the space of ML strategies and develop algorithms for predicting atomic-pairwise nuclear magnetic resonance (NMR) properties in molecules. Using an open-source dataset, we worked with Kaggle to design and host a 3-month competition which received 47,800 ML model predictions from 2,700 teams in 84 countries. Within 3 weeks, the Kaggle community produced models with comparable accuracy to our best previously published "in-house" efforts. A meta-ensemble model constructed as a linear combination of the top predictions has a prediction accuracy which exceeds that of any individual model, 7-19x better than our previous state-of-the-art. The results highlight the potential of transformer architectures for predicting quantum mechanical (QM) molecular properties.

Michael O'Connor, Simon J. Bennie, Helen M. Deeks et al.

As molecular scientists have made progress in their ability to engineer nano-scale molecular structure, we are facing new challenges in our ability to engineer molecular dynamics (MD) and flexibility. Dynamics at the molecular scale differs from the familiar mechanics of everyday objects, because it involves a complicated, highly correlated, and three-dimensional many-body dynamical choreography which is often non-intuitive even for highly trained researchers. We recently described how interactive molecular dynamics in virtual reality (iMD-VR) can help to meet this challenge, enabling researchers to manipulate real-time MD simulations of flexible structures in 3D. In this article, we outline efforts to extend immersive technologies to the molecular sciences, and we introduce 'Narupa', a flexible, open-source, multi-person iMD-VR software framework which enables groups of researchers to simultaneously cohabit real-time simulation environments to interactively visualize and manipulate the dynamics of molecular structures with atomic-level precision. We outline several application domains where iMD-VR is facilitating research, communication, and creative approaches within the molecular sciences, including training machines to learn reactive potential energy surfaces (PESs), biomolecular conformational sampling, protein-ligand binding, reaction discovery using 'on-the-fly' quantum chemistry, and transport dynamics in materials. We touch on iMD-VR's various cognitive and perceptual affordances, and how these provide research insight for molecular systems. By synergistically combining human spatial reasoning and design insight with computational automation, technologies like iMD-VR have the potential to improve our ability to understand, engineer, and communicate microscopic dynamical behavior, offering the potential to usher in a new paradigm for engineering molecules and nano-architectures.

Rachel Freire, Becca Rose Glowacki, Rhoslyn Roebuck Williams et al.

As VR finds increasing application in scientific research domains like nanotechnology and biochemistry, we are beginning to better understand the domains in which it brings the most benefit, as well as the gestures and form factors that are most useful for specific applications. Here we describe Open-source Mudra Gloves for Virtual Reality (OMG-VR): etextile gloves designed to facilitate research scientists and students carrying out detailed and complex manipulation of simulated 3d molecular objects in VR. The OMG-VR is designed to sense when a user pinches together their thumb and index finger, or thumb and middle finger, forming a "mudra" position. Tests show that they provide good positional tracking of the point at which a pinch takes place, require no calibration, and are sufficiently accurate and robust to enable scientists to accomplish a range of tasks that involve complex spatial manipulation of molecules. The open source design offers a promising alternative to existing controllers and more costly commercial VR data gloves.

David R. Glowacki, Rhoslyn Roebuck Williams, Olivia M. Maynard et al.

With a growing body of research highlighting the therapeutic potential of experiential phenomenology which diminishes egoic identity and increases one's sense of connectedness, there is significant interest in how to elicit such 'self-transcendent experiences' (STEs) in laboratory contexts. Psychedelic drugs (YDs) have proven particularly effective in this respect, producing subjective phenomenology which reliably elicits intense STEs. With virtual reality (VR) emerging as a powerful tool for constructing new perceptual environments, we describe a VR framework called 'Isness-distributed' (Isness-D) which harnesses the unique affordances of distributed multi-person VR to blur conventional self-other boundaries. Within Isness-D, groups of participants co-habit a shared virtual space, collectively experiencing their bodies as luminous energetic essences with diffuse spatial boundaries. It enables moments of 'energetic coalescence', a new class of embodied phenomenological intersubjective experience where bodies can fluidly merge, enabling participants to have an experience of including multiple others within their self-representation. To evaluate Isness-D, we adopted a citizen science approach, coordinating an international network of Isness-D 'nodes'. We analyzed the results (N = 58) using 4 different self-report scales previously applied to analyze subjective YD phenomenology (the inclusion of community in self scale, ego-dissolution inventory, communitas scale, and the MEQ30 mystical experience questionnaire). Despite the complexities associated with a distributed experiment like this, the Isness-D scores on all 4 scales were statistically indistinguishable from recently published YD studies, demonstrating that distributed VR can be used to design intersubjective STEs where people dissolve their sense of self in the connection to others.

Rhoslyn Roebuck Williams, Xan Varcoe, Becca R. Glowacki et al.

During VR demos we have performed over last few years, many participants (in the absence of any haptic feedback) have commented on their perceived ability to 'feel' differences between simulated molecular objects. The mechanisms for such 'feeling' are not entirely clear: observing from outside VR, one can see that there is nothing physical for participants to 'feel'. Here we outline exploratory user studies designed to evaluate the extent to which participants can distinguish quantitative differences in the flexibility of VR-simulated molecular objects. The results suggest that an individual's capacity to detect differences in molecular flexibility is enhanced when they can interact with and manipulate the molecules, as opposed to merely observing the same interaction. Building on these results, we intend to carry out further studies investigating humans' ability to sense quantitative properties of VR simulations without haptic technology.

David R. Glowacki, Mark D. Wonnacott, Rachel Freire et al.

Studies combining psychotherapy with psychedelic drugs (PsiDs) have demonstrated positive outcomes that are often associated with PsiDs' ability to induce 'mystical-type' experiences (MTEs) - i.e., subjective experiences whose characteristics include a sense of connectedness, transcendence, and ineffability. We suggest that both PsiDs and virtual reality can be situated on a broader spectrum of psychedelic technologies. To test this hypothesis, we used concepts, methods, and analysis strategies from PsiD research to design and evaluate 'Isness', a multi-person VR journey where participants experience the collective emergence, fluctuation, and dissipation of their bodies as energetic essences. A study (N=57) analyzing participant responses to a commonly used PsiD experience questionnaire (MEQ30) indicates that Isness participants had MTEs comparable to those reported in double-blind clinical studies after high doses of psilocybin & LSD. Within a supportive setting and conceptual framework, VR phenomenology can create the conditions for MTEs from which participants derive insight and meaning.

Lisa May Thomas, Helen M. Deeks, Alex J. Jones et al.

In most VR experiences, the visual sense dominates other modes of sensory input, encouraging non-visual senses to respond as if the visual were real. The simulated visual world thus becomes a sort of felt actuality, where the 'actual' physical body and environment can 'drop away', opening up possibilities for designing entirely new kinds of experience. Most VR experiences place visual sensory input (of the simulated environment) in the perceptual foreground, and the physical body in the background. In what follows, we discuss methods for resolving the apparent tension which arises from VR's prioritization of visual perception. We specifically aim to understand how somatic techniques encouraging participants to 'attend to their attention' enable them to access more subtle aspects of sensory phenomena in a VR experience, bound neither by rigid definitions of vision-based virtuality nor body-based corporeality. During a series of workshops, we implemented experimental somatic-dance practices to better understand perceptual and imaginative subtleties that arise for participants whilst they are embedded in a multi-person VR framework. Our preliminary observations suggest that somatic methods can be used to design VR experiences which enable (i) a tactile quality or felt sense of phenomena in the virtual environment (VE), (ii) lingering impacts on participant imagination even after the VR headset is taken off, and (iii) an expansion of imaginative potential.

Robert E. Arbon, Alex J. Jones, Lars A. Bratholm et al.

Translating the complex, multi-dimensional data from simulations of biomolecules to intuitive knowledge is a major challenge in computational chemistry and biology. The so-called "free energy landscape" is amongst the most fundamental concepts used by scientists to understand both static and dynamic properties of biomolecular systems. In this paper we use Markov models to design a strategy for mapping features of this landscape to sonic parameters, for use in conjunction with visual display techniques such as structural animations and free energy diagrams.

Michael O Connor, Helen M. Deeks, Edward Dawn et al.

We describe a framework for interactive molecular dynamics in a multiuser virtual reality environment, combining rigorous cloud-mounted physical atomistic simulation with commodity virtual reality hardware, which we have made accessible to readers (see isci.itch.io/nsb-imd). It allows users to visualize and sample, with atomic-level precision, the structures and dynamics of complex molecular structures 'on the fly', and to interact with other users in the same virtual environment. A series of controlled studies, wherein participants were tasked with a range of molecular manipulation goals (threading methane through a nanotube, changing helical screw-sense, and tying a protein knot), quantitatively demonstrate that users within the interactive VR environment can complete sophisticated molecular modelling tasks more quickly than they can using conventional interfaces, especially for molecular pathways and structural transitions whose conformational choreographies are intrinsically 3d. This framework should accelerate progress in nanoscale molecular engineering areas such as drug development, synthetic biology, and catalyst design. More broadly, our findings highlight VR's potential in scientific domains where 3d dynamics matter, spanning research and education.