Huanbo Sun

Huanbo Sun, Katherine J. Kuchenbecker, Georg Martius

Vision-based haptic sensors have emerged as a promising approach to robotic touch due to affordable high-resolution cameras and successful computer-vision techniques. However, their physical design and the information they provide do not yet meet the requirements of real applications. We present a robust, soft, low-cost, vision-based, thumb-sized 3D haptic sensor named Insight: it continually provides a directional force-distribution map over its entire conical sensing surface. Constructed around an internal monocular camera, the sensor has only a single layer of elastomer over-molded on a stiff frame to guarantee sensitivity, robustness, and soft contact. Furthermore, Insight is the first system to combine photometric stereo and structured light using a collimator to detect the 3D deformation of its easily replaceable flexible outer shell. The force information is inferred by a deep neural network that maps images to the spatial distribution of 3D contact force (normal and shear). Insight has an overall spatial resolution of 0.4 mm, force magnitude accuracy around 0.03 N, and force direction accuracy around 5 degrees over a range of 0.03--2 N for numerous distinct contacts with varying contact area. The presented hardware and software design concepts can be transferred to a wide variety of robot parts.

Huanbo Sun, Georg Martius

Haptic feedback is important to make robots more dexterous and effective in unstructured environments. High-resolution haptic sensors are still not widely available, and their application is often bound by the resolution-robustness dilemma. A route towards high-resolution and robust skin embeds a few sensor units (taxels) into a flexible surface material and uses signal processing to achieve sensing with super-resolution accuracy. We propose a theory for geometric super-resolution to guide the development of haptic sensors of this kind and link it to machine learning techniques for signal processing. This theory is based on sensor isolines and allows us to predict force sensitivity and accuracy in contact position and force magnitude as a spatial quantity. We evaluate the influence of different factors, such as elastic properties of the material, structure design, and transduction methods, using finite element simulations and by implementing real sensors. We empirically determine sensor isolines and validate the theory in two custom-built sensors with barometric units for 1D and 2D measurement surfaces. Using machine learning methods for the inference of contact information, our sensors obtain an unparalleled average super-resolution factor of over 100 and 1200, respectively. Our theory can guide future haptic sensor designs and inform various design choices.

Huanbo Sun, Goerg Martius

Haptic sensation is an important modality for interacting with the real world. This paper proposes a general framework of inferring haptic forces on the surface of a 3D structure from internal deformations using a small number of physical sensors instead of employing dense sensor arrays. Using machine learning techniques, we optimize the sensor number and their placement and are able to obtain high-precision force inference for a robotic limb using as few as 9 sensors. For the optimal and sparse placement of the measurement units (strain gauges), we employ data-driven methods based on data obtained by finite element simulation. We compare data-driven approaches with model-based methods relying on geometric distance and information criteria such as Entropy and Mutual Information. We validate our approach on a modified limb of the Poppy robot and obtain 8 mm localization precision.