Martin Bauer, Martins Bruveris, Peter W. Michor
We study reparametrization invariant Sobolev metrics on spaces of regular curves. We discuss their completeness properties and the resulting usability for applications in shape analysis. In particular, we will argue, that the development of efficient numerical methods for higher order Sobolev type metrics is an extremely desirable goal.
Martin Bauer, Martins Bruveris, Stephen Marsland et al.
Metrics on shape space are used to describe deformations that take one shape to another, and to determine a distance between them. We study a family of metrics on the space of curves, that includes several recently proposed metrics, for which the metrics are characterised by mappings into vector spaces where geodesics can be easily computed. This family consists of Sobolev-type Riemannian metrics of order one on the space $\text{Imm}(S^1,\mathbb R^2)$ of parametrized plane curves and the quotient space $\text{Imm}(S^1,\mathbb R^2)/\text{Diff}(S^1)$ of unparametrized curves. For the space of open parametrized curves we find an explicit formula for the geodesic distance and show that the sectional curvatures vanish on the space of parametrized and are non-negative on the space of unparametrized open curves. For the metric, which is induced by the "R-transform", we provide a numerical algorithm that computes geodesics between unparameterised, closed curves, making use of a constrained formulation that is implemented numerically using the RATTLE algorithm. We illustrate the algorithm with some numerical tests that demonstrate it's efficiency and robustness.
Emmanuel Hartman, Yashil Sukurdeep, Eric Klassen et al.
This paper introduces a set of numerical methods for Riemannian shape analysis of 3D surfaces within the setting of invariant (elastic) second-order Sobolev metrics. More specifically, we address the computation of geodesics and geodesic distances between parametrized or unparametrized immersed surfaces represented as 3D meshes. Building on this, we develop tools for the statistical shape analysis of sets of surfaces, including methods for estimating Karcher means and performing tangent PCA on shape populations, and for computing parallel transport along paths of surfaces. Our proposed approach fundamentally relies on a relaxed variational formulation for the geodesic matching problem via the use of varifold fidelity terms, which enable us to enforce reparametrization independence when computing geodesics between unparametrized surfaces, while also yielding versatile algorithms that allow us to compare surfaces with varying sampling or mesh structures. Importantly, we demonstrate how our relaxed variational framework can be extended to tackle partially observed data. The different benefits of our numerical pipeline are illustrated over various examples, synthetic and real.
Martin Bauer, Martins Bruveris
We present a new approach for matching regular surfaces in a Riemannian setting. We use a Sobolev type metric on deformation vector fields which form the tangent bundle to the space of surfaces. In this article we compare our approach with the diffeomorphic matching framework. In the latter approach a deformation is prescribed on the ambient space, which then drags along an embedded surface. In contrast our metric is defined directly on the deformation vector field and can therefore be called an inner metric. We also show how to discretize the corresponding geodesic equation and compute the gradient of the cost functional using finite elements.
Martin Bauer, Sarang Joshi, Klas Modin
In this article we explore an algorithm for diffeomorphic random sampling of nonuniform probability distributions on Riemannian manifolds. The algorithm is based on optimal information transport (OIT)---an analogue of optimal mass transport (OMT). Our framework uses the deep geometric connections between the Fisher-Rao metric on the space of probability densities and the right-invariant information metric on the group of diffeomorphisms. The resulting sampling algorithm is a promising alternative to OMT, in particular as our formulation is semi-explicit, free of the nonlinear Monge--Ampere equation. Compared to Markov Chain Monte Carlo methods, we expect our algorithm to stand up well when a large number of samples from a low dimensional nonuniform distribution is needed.
Martin Bauer, Martins Bruveris, Philipp Harms et al.
Statistical shape analysis can be done in a Riemannian framework by endowing the set of shapes with a Riemannian metric. Sobolev metrics of order two and higher on shape spaces of parametrized or unparametrized curves have several desirable properties not present in lower order metrics, but their discretization is still largely missing. In this paper, we present algorithms to numerically solve the geodesic initial and boundary value problems for these metrics. The combination of these algorithms enables one to compute Karcher means in a Riemannian gradient-based optimization scheme and perform principal component analysis and clustering. Our framework is sufficiently general to be applicable to a wide class of metrics. We demonstrate the effectiveness of our approach by analyzing a collection of shapes representing HeLa cell nuclei.
Emmanuel Hartman, Emery Pierson, Martin Bauer et al.
We present Basis Restricted Elastic Shape Analysis (BaRe-ESA), a novel Riemannian framework for human body scan representation, interpolation and extrapolation. BaRe-ESA operates directly on unregistered meshes, i.e., without the need to establish prior point to point correspondences or to assume a consistent mesh structure. Our method relies on a latent space representation, which is equipped with a Riemannian (non-Euclidean) metric associated to an invariant higher-order metric on the space of surfaces. Experimental results on the FAUST and DFAUST datasets show that BaRe-ESA brings significant improvements with respect to previous solutions in terms of shape registration, interpolation and extrapolation. The efficiency and strength of our model is further demonstrated in applications such as motion transfer and random generation of body shape and pose.
Haocheng Dai, Martin Bauer, P. Thomas Fletcher et al.
The goal of diffusion-weighted magnetic resonance imaging (DWI) is to infer the structural connectivity of an individual subject's brain in vivo. To statistically study the variability and differences between normal and abnormal brain connectomes, a mathematical model of the neural connections is required. In this paper, we represent the brain connectome as a Riemannian manifold, which allows us to model neural connections as geodesics. This leads to the challenging problem of estimating a Riemannian metric that is compatible with the DWI data, i.e., a metric such that the geodesic curves represent individual fiber tracts of the connectomics. We reduce this problem to that of solving a highly nonlinear set of partial differential equations (PDEs) and study the applicability of convolutional encoder-decoder neural networks (CEDNNs) for solving this geometrically motivated PDE. Our method achieves excellent performance in the alignment of geodesics with white matter pathways and tackles a long-standing issue in previous geodesic tractography methods: the inability to recover crossing fibers with high fidelity.
Martin Bauer, Thomas Fidler, Markus Grasmair
This article is concerned with the representation of curves by means of integral invariants. In contrast to the classical differential invariants they have the advantage of being less sensitive with respect to noise. The integral invariant most common in use is the circular integral invariant. A major drawback of this curve descriptor, however, is the absence of any uniqueness result for this representation. This article serves as a contribution towards closing this gap by showing that the circular integral invariant is injective in a neighbourhood of the circle. In addition, we provide a stability estimate valid on this neighbourhood. The proof is an application of Riesz-Schauder theory and the implicit function theorem in a Banach space setting.
Martin Bauer, Markus Grasmair, Clemens Kirisits
Optical flow is a powerful tool for the study and analysis of motion in a sequence of images. In this article we study a Horn-Schunck type spatio-temporal regularization functional for image sequences that have a non-Euclidean, time varying image domain. To that end we construct a Riemannian metric that describes the deformation and structure of this evolving surface. The resulting functional can be seen as natural geometric generalization of previous work by Weickert and Schnörr (2001) and Lefèvre and Baillet (2008) for static image domains. In this work we show the existence and wellposedness of the corresponding optical flow problem and derive necessary and sufficient optimality conditions. We demonstrate the functionality of our approach in a series of experiments using both synthetic and real data.
Emmanuel Hartman, Emery Pierson, Martin Bauer et al.
This paper introduces a new mathematical and numerical framework for surface analysis derived from the general setting of elastic Riemannian metrics on shape spaces. Traditionally, those metrics are defined over the infinite dimensional manifold of immersed surfaces and satisfy specific invariance properties enabling the comparison of surfaces modulo shape preserving transformations such as reparametrizations. The specificity of the approach we develop is to restrict the space of allowable transformations to predefined finite dimensional bases of deformation fields. These are estimated in a data-driven way so as to emulate specific types of surface transformations observed in a training set. The use of such bases allows to simplify the representation of the corresponding shape space to a finite dimensional latent space. However, in sharp contrast with methods involving e.g. mesh autoencoders, the latent space is here equipped with a non-Euclidean Riemannian metric precisely inherited from the family of aforementioned elastic metrics. We demonstrate how this basis restricted model can be then effectively implemented to perform a variety of tasks on surface meshes which, importantly, does not assume these to be pre-registered (i.e. with given point correspondences) or to even have a consistent mesh structure. We specifically validate our approach on human body shape and pose data as well as human face scans, and show how it generally outperforms state-of-the-art methods on problems such as shape registration, interpolation, motion transfer or random pose generation.
Martin Bauer, Facundo Mémoli, Tom Needham et al.
The Gromov-Wasserstein (GW) distance is a powerful tool for comparing metric measure spaces which has found broad applications in data science and machine learning. Driven by the need to analyze datasets whose objects have increasingly complex structure (such as node and edge-attributed graphs), several variants of GW distance have been introduced in the recent literature. With a view toward establishing a general framework for the theory of GW-like distances, this paper considers a vast generalization of the notion of a metric measure space: for an arbitrary metric space $Z$, we define a $Z$-network to be a measure space endowed with a kernel valued in $Z$. We introduce a method for comparing $Z$-networks by defining a generalization of GW distance, which we refer to as $Z$-Gromov-Wasserstein ($Z$-GW) distance. This construction subsumes many previously known metrics and offers a unified approach to understanding their shared properties. This paper demonstrates that the $Z$-GW distance defines a metric on the space of $Z$-networks which retains desirable properties of $Z$, such as separability, completeness, and geodesicity. Many of these properties were unknown for existing variants of GW distance that fall under our framework. Our focus is on foundational theory, but our results also include computable lower bounds and approximations of the distance which will be useful for practical applications.
Emmanuel Hartman, Nicolas Charon, Martin Bauer
This paper introduces self-supervised neural network models to tackle several fundamental problems in the field of 3D human body analysis and processing. First, we propose VariShaPE (Varifold Shape Parameter Estimator), a novel architecture for the retrieval of latent space representations of body shapes and poses. This network offers a fast and robust method to estimate the embedding of arbitrary unregistered meshes into the latent space. Second, we complement the estimation of latent codes with MoGeN (Motion Geometry Network) a framework that learns the geometry on the latent space itself. This is achieved by lifting the body pose parameter space into a higher dimensional Euclidean space in which body motion mini-sequences from a training set of 4D data can be approximated by simple linear interpolation. Using the SMPL latent space representation we illustrate how the combination of these network models, once trained, can be used to perform a variety of tasks with very limited computational cost. This includes operations such as motion interpolation, extrapolation and transfer as well as random shape and pose generation.
Kristen M. Campbell, Haocheng Dai, Zhe Su et al.
The structural network of the brain, or structural connectome, can be represented by fiber bundles generated by a variety of tractography methods. While such methods give qualitative insights into brain structure, there is controversy over whether they can provide quantitative information, especially at the population level. In order to enable population-level statistical analysis of the structural connectome, we propose representing a connectome as a Riemannian metric, which is a point on an infinite-dimensional manifold. We equip this manifold with the Ebin metric, a natural metric structure for this space, to get a Riemannian manifold along with its associated geometric properties. We then use this Riemannian framework to apply object-oriented statistical analysis to define an atlas as the Fréchet mean of a population of Riemannian metrics. This formulation ties into the existing framework for diffeomorphic construction of image atlases, allowing us to construct a multimodal atlas by simultaneously integrating complementary white matter structure details from DWMRI and cortical details from T1-weighted MRI. We illustrate our framework with 2D data examples of connectome registration and atlas formation. Finally, we build an example 3D multimodal atlas using T1 images and connectomes derived from diffusion tensors estimated from a subset of subjects from the Human Connectome Project.
Martin Bauer
For IoT to reach its full potential, the sharing and reuse of information in different applications and across verticals is of paramount importance. However, there are a plethora of IoT platforms using different representations, protocols and interaction patterns. To address this issue, the Fed4IoT project has developed an IoT virtualization platform that, on the one hand, integrates information from many different source platforms and, on the other hand, makes the information required by the respective users available in the target platform of choice. To enable this, information is translated into a common, neutral exchange format. The format of choice is NGSI-LD, which is being standardized by the ETSI Industry Specification Group on Context Information Management (ETSI ISG CIM). Thing Visors are the components that translate the source information to NGSI-LD, which is then delivered to the target platform and translated into the target format. ThingVisors can be implemented by hand, but this requires significant human effort, especially considering the heterogeneity of low level information produced by a multitude of sensors. Thus, supporting the human developer and, ideally, fully automating the process of extracting and enriching data and translating it to NGSI-LD is a crucial step. Machine learning is a promising approach for this, but it typically requires large amounts of hand-labelled data for training, an effort that makes it unrealistic in many IoT scenarios. A programmatic labelling approach called knowledge infusion that encodes expert knowledge is used for matching a schema or ontology extracted from the data with a target schema or ontology, providing the basis for annotating the data and facilitating the translation to NGSI-LD.
Amanpreet Singh, Martin Bauer, Sarang Joshi
Optimal Mass Transport (OMT) is a well studied problem with a variety of applications in a diverse set of fields ranging from Physics to Computer Vision and in particular Statistics and Data Science. Since the original formulation of Monge in 1781 significant theoretical progress been made on the existence, uniqueness and properties of the optimal transport maps. The actual numerical computation of the transport maps, particularly in high dimensions, remains a challenging problem. By Brenier's theorem, the continuous OMT problem can be reduced to that of solving a non-linear PDE of Monge-Ampere type whose solution is a convex function. In this paper, building on recent developments of input convex neural networks and physics informed neural networks for solving PDE's, we propose a Deep Learning approach to solve the continuous OMT problem. To demonstrate the versatility of our framework we focus on the ubiquitous density estimation and generative modeling tasks in statistics and machine learning. Finally as an example we show how our framework can be incorporated with an autoencoder to estimate an effective probabilistic generative model.
Kristen M. Campbell, Haocheng Dai, Zhe Su et al.
The structural connectome is often represented by fiber bundles generated from various types of tractography. We propose a method of analyzing connectomes by representing them as a Riemannian metric, thereby viewing them as points in an infinite-dimensional manifold. After equipping this space with a natural metric structure, the Ebin metric, we apply object-oriented statistical analysis to define an atlas as the Fréchet mean of a population of Riemannian metrics. We demonstrate connectome registration and atlas formation using connectomes derived from diffusion tensors estimated from a subset of subjects from the Human Connectome Project.
Emmanuel Hartman, Yashil Sukurdeep, Nicolas Charon et al.
Motivated by applications from computer vision to bioinformatics, the field of shape analysis deals with problems where one wants to analyze geometric objects, such as curves, while ignoring actions that preserve their shape, such as translations, rotations, or reparametrizations. Mathematical tools have been developed to define notions of distances, averages, and optimal deformations for geometric objects. One such framework, which has proven to be successful in many applications, is based on the square root velocity (SRV) transform, which allows one to define a computable distance between spatial curves regardless of how they are parametrized. This paper introduces a supervised deep learning framework for the direct computation of SRV distances between curves, which usually requires an optimization over the group of reparametrizations that act on the curves. The benefits of our approach in terms of computational speed and accuracy are illustrated via several numerical experiments.
Martin Bauer, Nicolas Charon, Philipp Harms et al.
Surface comparison and matching is a challenging problem in computer vision. While reparametrization-invariant Sobolev metrics provide meaningful elastic distances and point correspondences via the geodesic boundary value problem, solving this problem numerically tends to be difficult. Square root normal fields (SRNF) considerably simplify the computation of certain elastic distances between parametrized surfaces. Yet they leave open the issue of finding optimal reparametrizations, which induce elastic distances between unparametrized surfaces. This issue has concentrated much effort in recent years and led to the development of several numerical frameworks. In this paper, we take an alternative approach which bypasses the direct estimation of reparametrizations: we relax the geodesic boundary constraint using an auxiliary parametrization-blind varifold fidelity metric. This reformulation has several notable benefits. By avoiding altogether the need for reparametrizations, it provides the flexibility to deal with simplicial meshes of arbitrary topologies and sampling patterns. Moreover, the problem lends itself to a coarse-to-fine multi-resolution implementation, which makes the algorithm scalable to large meshes. Furthermore, this approach extends readily to higher-order feature maps such as square root curvature fields and is also able to include surface textures in the matching problem. We demonstrate these advantages on several examples, synthetic and real.
Yashil Sukurdeep, Martin Bauer, Nicolas Charon
This paper introduces a new mathematical formulation and numerical approach for the computation of distances and geodesics between immersed planar curves. Our approach combines the general simplifying transform for first-order elastic metrics that was recently introduced by Kurtek and Needham, together with a relaxation of the matching constraint using parametrization-invariant fidelity metrics. The main advantages of this formulation are that it leads to a simple optimization problem for discretized curves, and that it provides a flexible approach to deal with noisy, inconsistent or corrupted data. These benefits are illustrated via a few preliminary numerical results.
Erno Kovacs, Martin Bauer, Jaeho Kim et al.
Global IoT services (GIoTS) are combining locally available IoT resources with Cloud-based services. They are targeting world-wide services. GIoTS require interoperability between the locally installed heterogeneous IoT systems. Semantic processing is an important technology to enable data mediation as well as knowledge-based processing. This paper explains a system architecture for achieving world-wide semantic interoperability using international standards like oneM2M and the OMA NGSI-9/10 context interfaces (as used in the European Future Internet Platform FIWARE). Semantics also enables the use of Knowledge-based Semantic Processing Agents. Furthermore, we explain how semantic verification enables the testing of such complex systems.
Martin Bauer, Martins Bruveris, Nicolas Charon et al.
In this paper we study a class of Riemannian metrics on the space of unparametrized curves and develop a method to compute geodesics with given boundary conditions. It extends previous works on this topic in several important ways. The model and resulting matching algorithm integrate within one common setting both the family of $H^2$-metrics with constant coefficients and scale-invariant $H^2$-metrics on both open and closed immersed curves. These families include as particular cases the class of first-order elastic metrics. An essential difference with prior approaches is the way that boundary constraints are dealt with. By leveraging varifold-based similarity metrics we propose a relaxed variational formulation for the matching problem that avoids the necessity of optimizing over the reparametrization group. Furthermore, we show that we can also quotient out finite-dimensional similarity groups such as translation, rotation and scaling groups. The different properties and advantages are illustrated through numerical examples in which we also provide a comparison with related diffeomorphic methods used in shape registration.
Caleb Rottman, Martin Bauer, Klas Modin et al.
In this article we study the problem of thoracic image registration, in particular the estimation of complex anatomical deformations associated with the breathing cycle. Using the intimate link between the Riemannian geometry of the space of diffeomorphisms and the space of densities, we develop an image registration framework that incorporates both the fundamental law of conservation of mass as well as spatially varying tissue compressibility properties. By exploiting the geometrical structure, the resulting algorithm is computationally efficient, yet widely general.
Martin Bauer, Martins Bruveris, Philipp Harms et al.
Second order Sobolev metrics on the space of regular unparametrized planar curves have several desirable completeness properties not present in lower order metrics, but numerics are still largely missing. In this paper, we present algorithms to numerically solve the initial and boundary value problems for geodesics. The combination of these algorithms allows to compute Karcher means in a Riemannian gradient-based optimization scheme. Our framework has the advantage that the constants determining the weights of the zero, first, and second order terms of the metric can be chosen freely. Moreover, due to its generality, it could be applied to more general spaces of mapping. We demonstrate the effectiveness of our approach by analyzing a collection of shapes representing physical objects.
Martin Bauer, Martins Bruveris, Philipp Harms et al.
In the recent years, Riemannian shape analysis of curves and surfaces has found several applications in medical image analysis. In this paper we present a numerical discretization of second order Sobolev metrics on the space of regular curves in Euclidean space. This class of metrics has several desirable mathematical properties. We propose numerical solutions for the initial and boundary value problems of finding geodesics. These two methods are combined in a Riemannian gradient-based optimization scheme to compute the Karcher mean. We apply this to a study of the shape variation in HeLa cell nuclei and cycles of cardiac deformations, by computing means and principal modes of variations.
Martin Bauer, Markus Eslitzbichler, Markus Grasmair
Motions of virtual characters in movies or video games are typically generated by recording actors using motion capturing methods. Animations generated this way often need postprocessing, such as improving the periodicity of cyclic animations or generating entirely new motions by interpolation of existing ones. Furthermore, search and classification of recorded motions becomes more and more important as the amount of recorded motion data grows. In this paper, we will apply methods from shape analysis to the processing of animations. More precisely, we will use the by now classical elastic metric model used in shape matching, and extend it by incorporating additional inexact feature point information, which leads to an improved temporal alignment of different animations.