Johan Thunberg

Zuogong Yue, Johan Thunberg, Wei Pan et al.

Performing multiple experiments is common when learning internal mechanisms of complex systems. These experiments can include perturbations to parameters or external disturbances. A challenging problem is to efficiently incorporate all collected data simultaneously to infer the underlying dynamic network. This paper addresses the reconstruction of dynamic networks from heterogeneous datasets under the assumption that underlying networks share the same Boolean structure across all experiments. Parametric models for dynamical structure functions are derived to describe causal interactions between measured variables. Multiple datasets are integrated into one regression problem with additional demands of group sparsity to assure network sparsity and structure consistency. To acquire structured group sparsity, we propose a sampling-based method, together with extended versions of l1 methods and sparse Bayesian learning. The performance of the proposed methods is benchmarked in numerical simulation. In summary, this paper presents efficient methods on network reconstruction from multiple experiments, and reveals practical experience that could guide applications.

Zuogon Yue, Johan Thunberg, Lennart Ljung et al.

Network reconstruction of dynamical continuous-time (CT) systems is motivated by applications in many fields. Due to experimental limitations, especially in biology, data could be sampled at low frequencies, leading to significant challenges in network inference. We introduce the concept of "system aliasing" and characterize the minimal sampling frequency that allows reconstruction of CT systems from low sampled data. A test criterion is also proposed to check whether system aliasing is presented. With no system aliasing, the paper provides an algorithm to reconstruct dynamic network from data in the presence of noise. In addition, when there is system aliasing we perform studies that add additional prior information of the system such as sparsity. This paper opens new directions in modelling of network systems where samples have significant costs. Such tools are essential to process the available data in applications subject to current experimental limitations.

Johan Thunberg, Xiaoming Hu, Jorge Goncalves

This paper addresses the consensus problem and the formation problem on SE(3) in multi-agent systems with directed and switching interconnection topologies. Several control laws are introduced for the consensus problem. By a simple transformation, it is shown that the proposed control laws can be used for the formation problem. The design is first conducted on the kinematic level, where the velocities are the control laws. Then, for rigid bodies in space, the design is conducted on the dynamic level, where the torques and the forces are the control laws. On the kinematic level, first two control laws are introduced that explicitly use Euclidean transformations, then separate control laws are defined for the rotations and the translations. In the special case of purely rotational motion, the consensus problem is referred to as consensus on SO(3) or attitude synchronization. In this problem, for a broad class of local representations or parameterizations of SO(3), including the Axis-Angle Representation, the Rodrigues Parameters and the Modified Rodrigues Parameters, two types of control laws are presented that look structurally the same for any choice of local representation. For these two control laws we provide conditions on the initial rotations and the connectivity of the graph such that the system reaches consensus on SO(3). Among the contributions of this paper, there are conditions for when exponential rate of convergence occur. A theorem is provided showing that for any choice of local representation for the rotations, there is a change of coordinates such that the transformed system has a well known structure.

Zuogong Yue, Johan Thunberg, Lennart Ljung et al.

Dynamic network reconstruction has been shown to be challenging due to the requirements on sparse network structures and network identifiability. The direct parametric method (e.g., using ARX models) requires a large amount of parameters in model selection. Amongst the parametric models, only a restricted class can easily be used to address network sparsity without rendering the optimization problem intractable. To overcome these problems, this paper presents a state-space-based method, which significantly reduces the number of unknown parameters in model selection. Furthermore, we avoid various difficulties arising in gradient computation by using the Expectation Minimization (EM) algorithm instead. To enhance network sparsity, the prior distribution is constructed by using the Sparse Bayesian Learning (SBL) approach in the M-step. To solve the SBL problem, another EM algorithm is embedded, where we impose conditions on network identifiability in each iteration. In a sum, this paper provides a solution to reconstruct dynamic networks that avoids the difficulties inherent to gradient computation and simplifies the model selection.

Johan Thunberg, Johan Markdahl, Jorge Goncalves

In the multi-agent systems setting, this paper addresses continuous-time distributed synchronization of columns of rotation matrices. More precisely, k specific columns shall be synchronized and only the corresponding k columns of the relative rotations between the agents are assumed to be available for the control design. When one specific column is considered, the problem is equivalent to synchronization on the (d-1)-dimensional unit sphere and when all the columns are considered, the problem is equivalent to synchronization on SO(d). We design dynamic control laws for these synchronization problems. The control laws are based on the introduction of auxiliary variables in combination with a QR-factorization approach. The benefit of this QR-factorization approach is that we can decouple the dynamics for the k columns from the remaining d-k ones. Under the control scheme, the closed loop system achieves almost global convergence to synchronization for quasi-strong interaction graph topologies.

Zuogong Yue, Johan Thunberg, Lennart Ljung et al.

This paper addresses identification of sparse linear and noise-driven continuous-time state-space systems, i.e., the right-hand sides in the dynamical equations depend only on a subset of the states. The key assumption in this study, is that the sample rate is not high enough to directly infer the continuous time system from the data. This assumption is relevant in applications where sampling is expensive or requires human intervention (e.g., biomedicine applications). We propose an iterative optimization scheme with $l_1$-regularization, where the search directions are restricted those that decrease prediction error in each iteration. We provide numerical examples illustrating the proposed method; the method outperforms the least squares estimation for large noise.

Zuogon Yue, Johan Thunberg, Jorge Goncalves

This note addresses identification of the $A$-matrix in continuous time linear dynamical systems on state-space form. If this matrix is partially known or known to have a sparse structure, such knowledge can be used to simplify the identification. We begin by introducing some general conditions for solvability of the inverse problems for matrix exponential. Next, we introduce "system aliasing" as an issue in the identification of slow sampled systems. Such aliasing give rise to non-unique matrix logarithms. As we show, by imposing additional conditions on and prior knowledge about the $A$-matrix, the issue of system aliasing can, at least partially, be overcome. Under conditions on the sparsity and the norm of the $A$-matrix, it is identifiable up to a finite equivalence class.

Johan Thunberg, Florian Bernard

We propose a novel non-negative spherical relaxation for optimization problems over binary matrices with injectivity constraints, which in particular has applications in multi-matching and clustering. We relax respective binary matrix constraints to the (high-dimensional) non-negative sphere. To optimize our relaxed problem, we use a conditional power iteration method to iteratively improve the objective function, while at same time sweeping over a continuous scalar parameter that is (indirectly) related to the universe size (or number of clusters). Opposed to existing procedures that require to fix the integer universe size before optimization, our method automatically adjusts the analogous continuous parameter. Furthermore, while our approach shares similarities with spectral multi-matching and spectral clustering, our formulation has the strong advantage that we do not rely on additional post-processing procedures to obtain binary results. Our method shows compelling results in various multi-matching and clustering settings, even when compared to methods that use the ground truth universe size (or number of clusters).

Jianbo Wang, Galina Sidorenko, Johan Thunberg

Connected and automated vehicles (CAVs) have the potential to enhance driving safety, for example by enabling safe vehicle following and more efficient traffic scheduling. For such future deployments, safety requirements should be addressed, where the primary such are avoidance of vehicle collisions and substantial mitigating of harm when collisions are unavoidable. However, conservative worst-case-based control strategies come at the price of reduced flexibility and may compromise overall performance. In light of this, we investigate how Deep Reinforcement Learning (DRL) can be leveraged to improve safety in multi-vehicle-following scenarios involving emergency braking. Specifically, we investigate how DRL with vehicle-to-vehicle communication can be used to ethically select an emergency breaking profile in scenarios where overall, or collective, three-vehicle harm reduction or collision avoidance shall be obtained instead of single-vehicle such. As an algorithm, we provide a hybrid approach that combines DRL with a previously published method based on analytical expressions for selecting optimal constant deceleration. By combining DRL with the previous method, the proposed hybrid approach increases the reliability compared to standalone DRL, while achieving superior performance in terms of overall harm reduction and collision avoidance.

Florian Bernard, Daniel Cremers, Johan Thunberg

We address the non-convex optimisation problem of finding a sparse matrix on the Stiefel manifold (matrices with mutually orthogonal columns of unit length) that maximises (or minimises) a quadratic objective function. Optimisation problems on the Stiefel manifold occur for example in spectral relaxations of various combinatorial problems, such as graph matching, clustering, or permutation synchronisation. Although sparsity is a desirable property in such settings, it is mostly neglected in spectral formulations since existing solvers, e.g. based on eigenvalue decomposition, are unable to account for sparsity while at the same time maintaining global optimality guarantees. We fill this gap and propose a simple yet effective sparsity-promoting modification of the Orthogonal Iteration algorithm for finding the dominant eigenspace of a matrix. By doing so, we can guarantee that our method finds a Stiefel matrix that is globally optimal with respect to the quadratic objective function, while in addition being sparse. As a motivating application we consider the task of permutation synchronisation, which can be understood as a constrained clustering problem that has particular relevance for matching multiple images or 3D shapes in computer vision, computer graphics, and beyond. We demonstrate that the proposed approach outperforms previous methods in this domain.

Maolin Gao, Zorah Lähner, Johan Thunberg et al.

Finding correspondences between shapes is a fundamental problem in computer vision and graphics, which is relevant for many applications, including 3D reconstruction, object tracking, and style transfer. The vast majority of correspondence methods aim to find a solution between pairs of shapes, even if multiple instances of the same class are available. While isometries are often studied in shape correspondence problems, they have not been considered explicitly in the multi-matching setting. This paper closes this gap by proposing a novel optimisation formulation for isometric multi-shape matching. We present a suitable optimisation algorithm for solving our formulation and provide a convergence and complexity analysis. Our algorithm obtains multi-matchings that are by construction provably cycle-consistent. We demonstrate the superior performance of our method on various datasets and set the new state-of-the-art in isometric multi-shape matching.

Florian Bernard, Johan Thunberg, Paul Swoboda et al.

The matching of multiple objects (e.g. shapes or images) is a fundamental problem in vision and graphics. In order to robustly handle ambiguities, noise and repetitive patterns in challenging real-world settings, it is essential to take geometric consistency between points into account. Computationally, the multi-matching problem is difficult. It can be phrased as simultaneously solving multiple (NP-hard) quadratic assignment problems (QAPs) that are coupled via cycle-consistency constraints. The main limitations of existing multi-matching methods are that they either ignore geometric consistency and thus have limited robustness, or they are restricted to small-scale problems due to their (relatively) high computational cost. We address these shortcomings by introducing a Higher-order Projected Power Iteration method, which is (i) efficient and scales to tens of thousands of points, (ii) straightforward to implement, (iii) able to incorporate geometric consistency, (iv) guarantees cycle-consistent multi-matchings, and (iv) comes with theoretical convergence guarantees. Experimentally we show that our approach is superior to existing methods.

Florian Bernard, Johan Thunberg, Jorge Goncalves et al.

In this work we study permutation synchronisation for the challenging case of partial permutations, which plays an important role for the problem of matching multiple objects (e.g. images or shapes). The term synchronisation refers to the property that the set of pairwise matchings is cycle-consistent, i.e. in the full matching case all compositions of pairwise matchings over cycles must be equal to the identity. Motivated by clustering and matrix factorisation perspectives of cycle-consistency, we derive an algorithm to tackle the permutation synchronisation problem based on non-negative factorisations. In order to deal with the inherent non-convexity of the permutation synchronisation problem, we use an initialisation procedure based on a novel rotation scheme applied to the solution of the spectral relaxation. Moreover, this rotation scheme facilitates a convenient Euclidean projection to obtain a binary solution after solving our relaxed problem. In contrast to state-of-the-art methods, our approach is guaranteed to produce cycle-consistent results. We experimentally demonstrate the efficacy of our method and show that it achieves better results compared to existing methods.

Johan Thunberg, Florian Bernard, Jorge Goncalves

This paper addresses the problem of synchronizing orthogonal matrices over directed graphs. For synchronized transformations (or matrices), composite transformations over loops equal the identity. We formulate the synchronization problem as a least-squares optimization problem with nonlinear constraints. The synchronization problem appears as one of the key components in applications ranging from 3D-localization to image registration. The main contributions of this work can be summarized as the introduction of two novel algorithms; one for symmetric graphs and one for graphs that are possibly asymmetric. Under general conditions, the former has guaranteed convergence to the solution of a spectral relaxation to the synchronization problem. The latter is stable for small step sizes when the graph is quasi-strongly connected. The proposed methods are verified in numerical simulations.

Florian Bernard, Frank R. Schmidt, Johan Thunberg et al.

We propose a combinatorial solution for the problem of non-rigidly matching a 3D shape to 3D image data. To this end, we model the shape as a triangular mesh and allow each triangle of this mesh to be rigidly transformed to achieve a suitable matching to the image. By penalising the distance and the relative rotation between neighbouring triangles our matching compromises between image and shape information. In this paper, we resolve two major challenges: Firstly, we address the resulting large and NP-hard combinatorial problem with a suitable graph-theoretic approach. Secondly, we propose an efficient discretisation of the unbounded 6-dimensional Lie group SE(3). To our knowledge this is the first combinatorial formulation for non-rigid 3D shape-to-image matching. In contrast to existing local (gradient descent) optimisation methods, we obtain solutions that do not require a good initialisation and that are within a bound of the optimal solution. We evaluate the proposed method on the two problems of non-rigid 3D shape-to-shape and non-rigid 3D shape-to-image registration and demonstrate that it provides promising results.

Florian Bernard, Luis Salamanca, Johan Thunberg et al.

The reconstruction of an object's shape or surface from a set of 3D points plays an important role in medical image analysis, e.g. in anatomy reconstruction from tomographic measurements or in the process of aligning intra-operative navigation and preoperative planning data. In such scenarios, one usually has to deal with sparse data, which significantly aggravates the problem of reconstruction. However, medical applications often provide contextual information about the 3D point data that allow to incorporate prior knowledge about the shape that is to be reconstructed. To this end, we propose the use of a statistical shape model (SSM) as a prior for surface reconstruction. The SSM is represented by a point distribution model (PDM), which is associated with a surface mesh. Using the shape distribution that is modelled by the PDM, we formulate the problem of surface reconstruction from a probabilistic perspective based on a Gaussian Mixture Model (GMM). In order to do so, the given points are interpreted as samples of the GMM. By using mixture components with anisotropic covariances that are "oriented" according to the surface normals at the PDM points, a surface-based fitting is accomplished. Estimating the parameters of the GMM in a maximum a posteriori manner yields the reconstruction of the surface from the given data points. We compare our method to the extensively used Iterative Closest Points method on several different anatomical datasets/SSMs (brain, femur, tibia, hip, liver) and demonstrate superior accuracy and robustness on sparse data.

Florian Bernard, Peter Gemmar, Frank Hertel et al.

Representing 3D shape deformations by linear models in high-dimensional space has many applications in computer vision and medical imaging, such as shape-based interpolation or segmentation. Commonly, using Principal Components Analysis a low-dimensional (affine) subspace of the high-dimensional shape space is determined. However, the resulting factors (the most dominant eigenvectors of the covariance matrix) have global support, i.e. changing the coefficient of a single factor deforms the entire shape. In this paper, a method to obtain deformation factors with local support is presented. The benefits of such models include better flexibility and interpretability as well as the possibility of interactively deforming shapes locally. For that, based on a well-grounded theoretical motivation, we formulate a matrix factorisation problem employing sparsity and graph-based regularisation terms. We demonstrate that for brain shapes our method outperforms the state of the art in local support models with respect to generalisation ability and sparse shape reconstruction, whereas for human body shapes our method gives more realistic deformations.

Johan Thunberg, Florian Bernard, Jorge Goncalves

Transitive consistency is an intrinsic property for collections of linear invertible transformations between Euclidean coordinate frames. In practice, when the transformations are estimated from data, this property is lacking. This work addresses the problem of synchronizing transformations that are not transitively consistent. Once the transformations have been synchronized, they satisfy the transitive consistency condition - a transformation from frame $A$ to frame $C$ is equal to the composite transformation of first transforming A to B and then transforming B to C. The coordinate frames correspond to nodes in a graph and the transformations correspond to edges in the same graph. Two direct or centralized synchronization methods are presented for different graph topologies; the first one for quasi-strongly connected graphs, and the second one for connected graphs. As an extension of the second method, an iterative Gauss-Newton method is presented, which is later adapted to the case of affine and Euclidean transformations. Two distributed synchronization methods are also presented for orthogonal matrices, which can be seen as distributed versions of the two direct or centralized methods; they are similar in nature to standard consensus protocols used for distributed averaging. When the transformations are orthogonal matrices, a bound on the optimality gap can be computed. Simulations show that the gap is almost right, even for noise large in magnitude. This work also contributes on a theoretical level by providing linear algebraic relationships for transitively consistent transformations. One of the benefits of the proposed methods is their simplicity - basic linear algebraic methods are used, e.g., the Singular Value Decomposition (SVD). For a wide range of parameter settings, the methods are numerically validated.

Florian Bernard, Johan Thunberg, Peter Gemmar et al.

The alignment of a set of objects by means of transformations plays an important role in computer vision. Whilst the case for only two objects can be solved globally, when multiple objects are considered usually iterative methods are used. In practice the iterative methods perform well if the relative transformations between any pair of objects are free of noise. However, if only noisy relative transformations are available (e.g. due to missing data or wrong correspondences) the iterative methods may fail. Based on the observation that the underlying noise-free transformations can be retrieved from the null space of a matrix that can directly be obtained from pairwise alignments, this paper presents a novel method for the synchronisation of pairwise transformations such that they are transitively consistent. Simulations demonstrate that for noisy transformations, a large proportion of missing data and even for wrong correspondence assignments the method delivers encouraging results.