Mohammad Keshavarzi

Mohammad Keshavarzi, Michael Zollhoefer, Allen Y. Yang et al. · berkeley

Remote telepresence via next-generation mixed reality platforms can provide higher levels of immersion for computer-mediated communications, allowing participants to engage in a wide spectrum of activities, previously not possible in 2D screen-based communication methods. However, as mixed reality experiences are limited to the local physical surrounding of each user, finding a common virtual ground where users can freely move and interact with each other is challenging. In this paper, we propose a novel mutual scene synthesis method that takes the participants' spaces as input, and generates a virtual synthetic scene that corresponds to the functional features of all participants' local spaces. Our method combines a mutual function optimization module with a deep-learning conditional scene augmentation process to generate a scene mutually and physically accessible to all participants of a mixed reality telepresence scenario. The synthesized scene can hold mutual walkable, sittable and workable functions, all corresponding to physical objects in the users' real environments. We perform experiments using the MatterPort3D dataset and conduct comparative user studies to evaluate the effectiveness of our system. Our results show that our proposed approach can be a promising research direction for facilitating contextualized telepresence systems for next-generation spatial computing platforms.

Mohammad Keshavarzi, Flaviano Christian Reyes, Ritika Shrivastava et al.

Indoor scene augmentation has become an emerging topic in the field of computer vision and graphics with applications in augmented and virtual reality. However, current state-of-the-art systems using deep neural networks require large datasets for training. In this paper we introduce GSACNet, a contextual scene augmentation system that can be trained with limited scene priors. GSACNet utilizes a novel parametric data augmentation method combined with a Graph Attention and Siamese network architecture followed by an Autoencoder network to facilitate training with small datasets. We show the effectiveness of our proposed system by conducting ablation and comparative studies with alternative systems on the Matterport3D dataset. Our results indicate that our scene augmentation outperforms prior art in scene synthesis with limited scene priors available.

Mohammad Keshavarzi, Oladapo Afolabi, Luisa Caldas et al.

The availability of rich 3D datasets corresponding to the geometrical complexity of the built environments is considered an ongoing challenge for 3D deep learning methodologies. To address this challenge, we introduce GenScan, a generative system that populates synthetic 3D scan datasets in a parametric fashion. The system takes an existing captured 3D scan as an input and outputs alternative variations of the building layout including walls, doors, and furniture with corresponding textures. GenScan is a fully automated system that can also be manually controlled by a user through an assigned user interface. Our proposed system utilizes a combination of a hybrid deep neural network and a parametrizer module to extract and transform elements of a given 3D scan. GenScan takes advantage of style transfer techniques to generate new textures for the generated scenes. We believe our system would facilitate data augmentation to expand the currently limited 3D geometry datasets commonly used in 3D computer vision, generative design, and general 3D deep learning tasks.

Mohammad Keshavarzi, Aakash Parikh, Xiyu Zhai et al.

Spatial computing experiences are constrained by the real-world surroundings of the user. In such experiences, augmenting virtual objects to existing scenes require a contextual approach, where geometrical conflicts are avoided, and functional and plausible relationships to other objects are maintained in the target environment. Yet, due to the complexity and diversity of user environments, automatically calculating ideal positions of virtual content that is adaptive to the context of the scene is considered a challenging task. Motivated by this problem, in this paper we introduce SceneGen, a generative contextual augmentation framework that predicts virtual object positions and orientations within existing scenes. SceneGen takes a semantically segmented scene as input, and outputs positional and orientational probability maps for placing virtual content. We formulate a novel spatial Scene Graph representation, which encapsulates explicit topological properties between objects, object groups, and the room. We believe providing explicit and intuitive features plays an important role in informative content creation and user interaction of spatial computing settings, a quality that is not captured in implicit models. We use kernel density estimation (KDE) to build a multivariate conditional knowledge model trained using prior spatial Scene Graphs extracted from real-world 3D scanned data. To further capture orientational properties, we develop a fast pose annotation tool to extend current real-world datasets with orientational labels. Finally, to demonstrate our system in action, we develop an Augmented Reality application, in which objects can be contextually augmented in real-time.

Mohammad Keshavarzi, Clayton Hutson, Chin-Yi Cheng et al.

Developing fully parametric building models for performance-based generative design tasks often requires proficiency in many advanced 3D modeling and visual programming, limiting its use for many building designers. Moreover, iterations of such models can be time-consuming tasks and sometimes limiting, as major changes in the layout design may result in remodeling the entire parametric definition. To address these challenges, we introduce a novel automated generative design system, which takes a basic floor plan sketch as an input and provides a parametric model prepared for multi-objective building optimization as output. Furthermore, the user-designer can assign various design variables for its desired building elements by using simple annotations in the drawing. The system would recognize the corresponding element and define variable constraints to prepare for a multi-objective optimization problem.

Mohammad Keshavarzi, Ardavan Bidgoli, Hans Kellner

Generative Design workflows have introduced alternative paradigms in the domain of computational design, allowing designers to generate large pools of valid solutions by defining a set of goals and constraints. However, analyzing and narrowing down the generated solution space, which usually consists of various high-dimensional properties, has been a major challenge in current generative workflows. By taking advantage of the interactive unbounded spatial exploration, and the visual immersion offered in virtual reality platforms, we propose V-Dream, a virtual reality generative analysis framework for exploring large-scale solution spaces. V-Dream proposes a hybrid search workflow in which a spatial stochastic search approach is combined with a recommender system allowing users to pick desired candidates and eliminate the undesired ones iteratively. In each cycle, V-Dream reorganizes the remaining options in clusters based on the defined features. Moreover, our framework allows users to inspect design solutions and evaluate their performance metrics in various hierarchical levels, assisting them in narrowing down the solution space through iterative cycles of search/select/re-clustering of the solutions in an immersive fashion. Finally, we present a prototype of our proposed framework, illustrating how users can navigate and narrow down desired solutions from a pool of over 16000 monitor stands generated by Autodesk's Dreamcatcher software.

Mohammad Keshavarzi, Allen Y. Yang, Woojin Ko et al.

Spatial computing experiences are physically constrained by the geometry and semantics of the local user environment. This limitation is elevated in remote multi-user interaction scenarios, where finding a common virtual ground physically accessible for all participants becomes challenging. Locating a common accessible virtual ground is difficult for the users themselves, particularly if they are not aware of the spatial properties of other participants. In this paper, we introduce a framework to generate an optimal mutual virtual space for a multi-user interaction setting where remote users' room spaces can have different layout and sizes. The framework further recommends movement of surrounding furniture objects that expand the size of the mutual space with minimal physical effort. Finally, we demonstrate the performance of our solution on real-world datasets and also a real HoloLens application. Results show the proposed algorithm can effectively discover optimal shareable space for multi-user virtual interaction and hence facilitate remote spatial computing communication in various collaborative workflows.

Mohammad Keshavarzi, Luisa Caldas, Luis Santos

This work introduces RadVR, a virtual reality tool for daylighting analysis that simultaneously combines qualitative assessments through immersive real-time renderings with quantitative physically correct daylighting simulations in a 6DOF virtual environment. By taking a 3D building model with material properties as input, RadVR allows users to (1) perform physically-based daylighting simulations via Radiance, (2) study sunlight in different hours-of-the-year, (3) interact with a 9-point-in-time matrix for the most representative times of the year, and (4) visualize, compare, and analyze daylighting simulation results. With an end-to-end workflow, RadVR integrates with 3D modeling software that is commonly used by building designers. Additionally, by conducting user experiments we compare the proposed system with DIVA for Rhino, a Radiance-based tool that uses conventional 2D-displays. The results show that RadVR can provide promising assistance in spatial understanding tasks, navigation, and sun position analysis in virtual reality.

Mohammad Keshavarzi, Michael Wu, Michael N. Chin et al.

Virtual and augmented reality communication platforms are seen as promising modalities for next-generation remote face-to-face interactions. Our study attempts to explore non-verbal communication features in relation to their conversation context for virtual and augmented reality mediated communication settings. We perform a series of user experiments, triggering nine conversation tasks in 4 settings, each containing corresponding non-verbal communication features. Our results indicate that conversation types which involve less emotional engagement are more likely to be acceptable in virtual reality and augmented reality settings with low-fidelity avatar representation, compared to scenarios that involve high emotional engagement or intellectually difficult discussions. We further systematically analyze and rank the impact of low-fidelity representation of micro-expressions, body scale, head pose, and hand gesture in affecting the user experience in one-on-one conversations, and validate that preserving micro-expression cues plays the most effective role in improving bi-directional conversations in future virtual and augmented reality settings.

Reza Abbasi-Asl, Mohammad Keshavarzi, Dorian Yao Chan

We study the performance of brain computer interface (BCI) system in a virtual reality (VR) environment and compare it to 2D regular displays. First, we design a headset that consists of three components: a wearable electroencephalography (EEG) device, a VR headset and an interface. Recordings of brain and behavior from human subjects, performing a wide variety of tasks using our device are collected. The tasks consist of object rotation or scaling in VR using either mental commands or facial expression (smile and eyebrow movement). Subjects are asked to repeat similar tasks on regular 2D monitor screens. The performance in 3-D virtual reality environment is considerably higher compared to the to the 2D screen. Particularly, the median number of success rate across trials for VR setting is double of that for the 2D setting (8 successful command in VR setting compared to 4 successful command in 2D screen in 1 minute trials). Our results suggest that the design of future BCI systems can remarkably benefit from the VR setting.