J. Frederico Carvalho

Joao Carvalho, An T. Le, Mark Baierl et al.

Learning priors on trajectory distributions can help accelerate robot motion planning optimization. Given previously successful plans, learning trajectory generative models as priors for a new planning problem is highly desirable. Prior works propose several ways on utilizing this prior to bootstrapping the motion planning problem. Either sampling the prior for initializations or using the prior distribution in a maximum-a-posterior formulation for trajectory optimization. In this work, we propose learning diffusion models as priors. We then can sample directly from the posterior trajectory distribution conditioned on task goals, by leveraging the inverse denoising process of diffusion models. Furthermore, diffusion has been recently shown to effectively encode data multimodality in high-dimensional settings, which is particularly well-suited for large trajectory dataset. To demonstrate our method efficacy, we compare our proposed method - Motion Planning Diffusion - against several baselines in simulated planar robot and 7-dof robot arm manipulator environments. To assess the generalization capabilities of our method, we test it in environments with previously unseen obstacles. Our experiments show that diffusion models are strong priors to encode high-dimensional trajectory distributions of robot motions.

João Carvalho, Davide Tateo, Fabio Muratore et al.

Reinforcement learning methods for robotics are increasingly successful due to the constant development of better policy gradient techniques. A precise (low variance) and accurate (low bias) gradient estimator is crucial to face increasingly complex tasks. Traditional policy gradient algorithms use the likelihood-ratio trick, which is known to produce unbiased but high variance estimates. More modern approaches exploit the reparametrization trick, which gives lower variance gradient estimates but requires differentiable value function approximators. In this work, we study a different type of stochastic gradient estimator: the Measure-Valued Derivative. This estimator is unbiased, has low variance, and can be used with differentiable and non-differentiable function approximators. We empirically evaluate this estimator in the actor-critic policy gradient setting and show that it can reach comparable performance with methods based on the likelihood-ratio or reparametrization tricks, both in low and high-dimensional action spaces.

Rika Antonova, Anastasiia Varava, Peiyang Shi et al.

Deformable objects present a formidable challenge for robotic manipulation due to the lack of canonical low-dimensional representations and the difficulty of capturing, predicting, and controlling such objects. We construct compact topological representations to capture the state of highly deformable objects that are topologically nontrivial. We develop an approach that tracks the evolution of this topological state through time. Under several mild assumptions, we prove that the topology of the scene and its evolution can be recovered from point clouds representing the scene. Our further contribution is a method to learn predictive models that take a sequence of past point cloud observations as input and predict a sequence of topological states, conditioned on target/future control actions. Our experiments with highly deformable objects in simulation show that the proposed multistep predictive models yield more precise results than those obtained from computational topology libraries. These models can leverage patterns inferred across various objects and offer fast multistep predictions suitable for real-time applications.

Isac Arnekvist, J. Frederico Carvalho, Danica Kragic et al.

Optimizing parameters with momentum, normalizing data values, and using rectified linear units (ReLUs) are popular choices in neural network (NN) regression. Although ReLUs are popular, they can collapse to a constant function and "die", effectively removing their contribution from the model. While some mitigations are known, the underlying reasons of ReLUs dying during optimization are currently poorly understood. In this paper, we consider the effects of target normalization and momentum on dying ReLUs. We find empirically that unit variance targets are well motivated and that ReLUs die more easily, when target variance approaches zero. To further investigate this matter, we analyze a discrete-time linear autonomous system, and show theoretically how this relates to a model with a single ReLU and how common properties can result in dying ReLU. We also analyze the gradients of a single-ReLU model to identify saddle points and regions corresponding to dying ReLU and how parameters evolve into these regions when momentum is used. Finally, we show empirically that this problem persist, and is aggravated, for deeper models including residual networks.

Anastasiia Varava, J. Frederico Carvalho, Danica Kragic et al.

In this work we propose algorithms to explicitly construct a conservative estimate of the configuration spaces of rigid objects in 2D and 3D. Our approach is able to detect compact path components and narrow passages in configuration space which are important for applications in robotic manipulation and path planning. Moreover, as we demonstrate, they are also applicable to identification of molecular cages in chemistry. Our algorithms are based on a decomposition of the resulting 3 and 6 dimensional configuration spaces into slices corresponding to a finite sample of fixed orientations in configuration space. We utilize dual diagrams of unions of balls and uniform grids of orientations to approximate the configuration space. We carry out experiments to evaluate the computational efficiency on a set of objects with different geometric features thus demonstrating that our approach is applicable to different object shapes. We investigate the performance of our algorithm by computing increasingly fine-grained approximations of the object's configuration space.

Samuele Tosatto, Joao Carvalho, Hany Abdulsamad et al.

Reinforcement learning (RL) algorithms still suffer from high sample complexity despite outstanding recent successes. The need for intensive interactions with the environment is especially observed in many widely popular policy gradient algorithms that perform updates using on-policy samples. The price of such inefficiency becomes evident in real-world scenarios such as interaction-driven robot learning, where the success of RL has been rather limited. We address this issue by building on the general sample efficiency of off-policy algorithms. With nonparametric regression and density estimation methods we construct a nonparametric Bellman equation in a principled manner, which allows us to obtain closed-form estimates of the value function, and to analytically express the full policy gradient. We provide a theoretical analysis of our estimate to show that it is consistent under mild smoothness assumptions and empirically show that our approach has better sample efficiency than state-of-the-art policy gradient methods.

Anastasiia Varava, J. Frederico Carvalho, Danica Kragic et al.

In this paper, we compute a conservative approximation of the path-connected components of the free space of a rigid object in a 2D workspace in order to solve two closely related problems: to determine whether there exists a collision-free path between two given configurations, and to verify whether an object can escape arbitrarily far from its initial configuration -- i.e., whether the object is caged. Furthermore, we consider two quantitative characteristics of the free space: the volume of path-connected components and the width of narrow passages. To address these problems, we decompose the configuration space into a set of two-dimensional slices, approximate them as two-dimensional alpha-complexes, and then study the relations between them. This significantly reduces the computational complexity compared to a direct approximation of the free space. We implement our algorithm and run experiments in a three-dimensional configuration space of a simple object showing runtime of less than 2 seconds.

J. Frederico Carvalho, Sergio Pequito, A. Pedro Aguiar et al.

In this paper, for linear time-invariant plants, where a collection of possible inputs and outputs are known a priori, we address the problem of determining the communication between outputs and inputs, i.e., information patterns, such that desired control objectives of the closed-loop system (for instance, stabilizability) through static output feedback may be ensured. We address this problem in the structural system theoretic context. To this end, given a specified structural pattern (locations of zeros/non-zeros) of the plant matrices, we introduce the concept of essential information patterns, i.e., communication patterns between outputs and inputs that satisfy the following conditions: (i) ensure arbitrary spectrum assignment of the closed-loop system, using static output feedback constrained to the information pattern, for almost all possible plant instances with the specified structural pattern; and (ii) any communication failure precludes the resulting information pattern from attaining the pole placement objective in (i). Subsequently, we study the problem of determining essential information patterns. First, we provide several necessary and sufficient conditions to verify whether a specified information pattern is essential or not. Further, we show that such conditions can be verified by resorting to algorithms with polynomial complexity (in the dimensions of the state, input and output). Although such verification can be performed efficiently, it is shown that the problem of determining essential information patterns is in general NP-hard. The main results of the paper are illustrated through examples.

Joao Carvalho, Sergio Pequito, A. Pedro Aguiar et al.

Motivated by the development and deployment of large-scale dynamical systems, often composed of geographically distributed smaller subsystems, we address the problem of verifying their controllability in a distributed manner. In this work we study controllability in the structural system theoretic sense, structural controllability. In other words, instead of focusing on a specific numerical system realization, we provide guarantees for equivalence classes of linear time-invariant systems on the basis of their structural sparsity patterns, i.e., location of zero/nonzero entries in the plant matrices. To this end, we first propose several necessary and/or sufficient conditions to ensure structural controllability of the overall system, on the basis of the structural patterns of the subsystems and their interconnections. The proposed verification criteria are shown to be efficiently implementable (i.e., with polynomial time complexity in the number of the state variables and inputs) in two important subclasses of interconnected dynamical systems: similar (i.e., every subsystem has the same structure), and serial (i.e., every subsystem outputs to at most one other subsystem). Secondly, we provide a distributed algorithm to verify structural controllability for interconnected dynamical systems. The proposed distributed algorithm is efficient and implementable at the subsystem level; the algorithm is iterative, based on communication among (physically) interconnected subsystems, and requires only local model and interconnection knowledge at each subsystem.