Marco Cavallo

Kevin Miao, Harsh Agrawal, Qihang Zhang et al. · apple-ml, gatech

Generating high-quality 3D content requires models capable of learning robust distributions of complex scenes and the real-world objects within them. Recent Gaussian-based 3D reconstruction techniques have achieved impressive results in recovering high-fidelity 3D assets from sparse input images by predicting 3D Gaussians in a feed-forward manner. However, these techniques often lack the extensive priors and expressiveness offered by Diffusion Models. On the other hand, 2D Diffusion Models, which have been successfully applied to denoise multiview images, show potential for generating a wide range of photorealistic 3D outputs but still fall short on explicit 3D priors and consistency. In this work, we aim to bridge these two approaches by introducing DSplats, a novel method that directly denoises multiview images using Gaussian Splat-based Reconstructors to produce a diverse array of realistic 3D assets. To harness the extensive priors of 2D Diffusion Models, we incorporate a pretrained Latent Diffusion Model into the reconstructor backbone to predict a set of 3D Gaussians. Additionally, the explicit 3D representation embedded in the denoising network provides a strong inductive bias, ensuring geometrically consistent novel view generation. Our qualitative and quantitative experiments demonstrate that DSplats not only produces high-quality, spatially consistent outputs, but also sets a new standard in single-image to 3D reconstruction. When evaluated on the Google Scanned Objects dataset, DSplats achieves a PSNR of 20.38, an SSIM of 0.842, and an LPIPS of 0.109.

Marco Cavallo, Angus G. Forbes

Despite advances in augmented reality (AR), the process of creating meaningful experiences with this technology is still extremely challenging. Due to different tracking implementations and hardware constraints, developing AR applications either requires low-level programming skills, or is done through specific authoring tools that largely sacrifice the possibility of customizing the AR experience. Existing development workflows also do not support previewing or simulating the AR experience, requiring a lengthy process of trial and error by which content creators deploy and physically test applications in each iteration. To mitigate these limitations, we propose CAVE-AR, a novel virtual reality system for authoring, simulating and debugging custom AR experiences. Available both as a standalone or a plug-in tool, CAVE-AR is based on the concept of representing in the same global reference system both in AR content and tracking information, mixing geographical information, architectural features, and sensor data to simulate the context of an AR experience. Thanks to its novel abstraction of existing tracking technologies, CAVE-AR operates independently of users' devices, and integrates with existing programming tools to provide maximum flexibility. Our VR application provides designers with ways to create and modify an AR application, even while others are in the midst of using it. CAVE-AR further allows the designer to track how users are behaving, preview what they are currently seeing, and interact with them through several different channels. To illustrate our proposed development workflow and demonstrate the advantages of our authoring system, we introduce two CAVEAR use cases in which an augmented reality application is created and tested. We compare the CAVE-AR workflow to traditional development methods and demonstrate the importance of simulation and live application debugging.

Marco Cavallo, Çağatay Demiralp

Data clustering is a common unsupervised learning method frequently used in exploratory data analysis. However, identifying relevant structures in unlabeled, high-dimensional data is nontrivial, requiring iterative experimentation with clustering parameters as well as data features and instances. The number of possible clusterings for a typical dataset is vast, and navigating in this vast space is also challenging. The absence of ground-truth labels makes it impossible to define an optimal solution, thus requiring user judgment to establish what can be considered a satisfiable clustering result. Data scientists need adequate interactive tools to effectively explore and navigate the large clustering space so as to improve the effectiveness of exploratory clustering analysis. We introduce \textit{Clustrophile~2}, a new interactive tool for guided clustering analysis. \textit{Clustrophile~2} guides users in clustering-based exploratory analysis, adapts user feedback to improve user guidance, facilitates the interpretation of clusters, and helps quickly reason about differences between clusterings. To this end, \textit{Clustrophile~2} contributes a novel feature, the Clustering Tour, to help users choose clustering parameters and assess the quality of different clustering results in relation to current analysis goals and user expectations. We evaluate \textit{Clustrophile~2} through a user study with 12 data scientists, who used our tool to explore and interpret sub-cohorts in a dataset of Parkinson's disease patients. Results suggest that \textit{Clustrophile~2} improves the speed and effectiveness of exploratory clustering analysis for both experts and non-experts.

Marco Cavallo, Çağatay Demiralp

Dimensionality reduction is a common method for analyzing and visualizing high-dimensional data across domains. Dimensionality-reduction algorithms involve complex optimizations and the reduced dimensions computed by these algorithms generally lack clear relation to the initial data dimensions. Therefore, interpreting and reasoning about dimensionality reductions can be difficult. In this work, we introduce two interaction techniques, \textit{forward projection} and \textit{backward projection}, for reasoning dynamically about scatter plots of dimensionally reduced data. We also contribute two related visualization techniques, \textit{prolines} and \textit{feasibility map} to facilitate and enrich the effective use of the proposed interactions, which we integrate in a new tool called \textit{Praxis}. To evaluate our techniques, we first analyze their time and accuracy performance across varying sample and dimension sizes. We then conduct a user study in which twelve data scientists use \textit{Praxis} so as to assess the usefulness of the techniques in performing exploratory data analysis tasks. Results suggest that our visual interactions are intuitive and effective for exploring dimensionality reductions and generating hypotheses about the underlying data.