Header logo is

Institute Talks

Safe Learning Control for Gaussian Process Models

Talk
  • 25 February 2020 • 14:00 15:00
  • Jonas Umlauft
  • MPI-IS Stuttgart, Heisenbergstr. 3, seminar room 2P4

Machine learning allows automated systems to identify structures and physical laws based on measured data, which is particularly useful in areas where an analytic derivation of a model is too tedious or not possible. Research in reinforcement learning led to impressive results and superhuman performance in well-structured tasks and games. However, to this day, data-driven models are rarely employed in the control of safety critical systems, because the success of a controller, which is based on these models, cannot be guaranteed. Therefore, the research presented in this talk analyzes the closed-loop behavior of learning control laws by means of rigorous proofs. More specifically, we propose a control law based on Gaussian process (GP) models, which actively avoids uncertainties in the state space and favors trajectories along the training data, where the system is well-known. We show that this behavior is optimal as it maximizes the probability of asymptotic stability. Additionally, we consider an event-triggered online learning control law, which safely explores an initially unknown system. It only takes new training data whenever the uncertainty in the system becomes too large. As the control law only requires a locally precise model, this novel learning strategy has a high data efficiency and provides safety guarantees.

Organizers: Sebastian Trimpe

Learning to Model 3D Human Face Geometry

Talk
  • 20 March 2020 • 11:00 12:00
  • Victoria Fernández Abrevaya
  • N3.022 (Aquarium)

In this talk I will present an overview of our recent works that learn deep geometric models for the 3D face from large datasets of scans. Priors for the 3D face are crucial for many applications: to constrain ill posed problems such as 3D reconstruction from monocular input, for efficient generation and animation of 3D virtual avatars, or even in medical domains such as recognition of craniofacial disorders. Generative models of the face have been widely used for this task, as well as deep learning approaches that have recently emerged as a robust alternative. Barring a few exceptions, most of these data-driven approaches were built from either a relatively limited number of samples (in the case of linear models of the shape), or by synthetic data augmentation (for deep-learning based approaches), mainly due to the difficulty in obtaining large-scale and accurate 3D scans of the face. Yet, there is a substantial amount of 3D information that can be gathered when considering publicly available datasets that have been captured over the last decade. I will discuss here our works that tackle the challenges of building rich geometric models out of these large and varied datasets, with the goal of modeling the facial shape, expression (i.e. motion) or geometric details. Concretely, I will talk about (1) an efficient and fully automatic approach for registration of large datasets of 3D faces in motion; (2) deep learning methods for modeling the facial geometry that can disentangle the shape and expression aspects of the face; and (3) a multi-modal learning approach for capturing geometric details from images in-the-wild, by simultaneously encoding both facial surface normal and natural image information.

Organizers: Jinlong Yang

Electro-active Ionic Elastomers

Talk
  • 23 March 2020 • 11:00 12:00
  • Prof. Antal Jákli
  • 2P04

Motivated by the low voltage driven actuation of ionic Electroactive Polymers (iEAPs) [1] [2], recently we began investigating ionic elastomers. In this talk I will discuss the preparation, physical characterization and electric bending actuation properties of two novel ionic elastomers; ionic polymer electrolyte membranes (iPEM)[3], and ionic liquid crystal elastomers (iLCE).[4] Both materials can be actuated by low frequency AC or DC voltages of less than 1 V. The bending actuation properties of the iPEMs are outperforming most of the well-developed iEAPs, and the not optimized first iLCEs are already comparable to them. Ionic liquid crystal elastomers also exhibit superior features, such as the alignment dependent actuation, which offers the possibility of pre-programed actuation pattern at the level of cross-linking process. Additionally, multiple (thermal, optical and electric) actuations are also possible. I will also discuss issues with compliant electrodes and possible soft robotic applications. [1] Y. Bar-Cohen, Electroactive Polyer Actuators as Artficial Muscles: Reality, Potential and Challenges, SPIE Press, Bellingham, 2004. [2] O. Kim, S. J. Kim, M. J. Park, Chem. Commun. 2018, 54, 4895. [3] C. P. H. Rajapaksha, C. Feng, C. Piedrahita, J. Cao, V. Kaphle, B. Lüssem, T. Kyu, A. Jákli, Macromol. Rapid Commun. 2020, in print. [4] C. Feng, C. P. H. Rajapaksha, J. M. Cedillo, C. Piedrahita, J. Cao, V. Kaphle, B. Lussem, T. Kyu, A. I. Jákli, Macromol. Rapid Commun. 2019, 1900299.

Biomechanical models and functional anatomy of the horse body

Talk
  • 23 March 2020 • 12:00 12:45
  • Elin Herlund
  • N3.022 (Aquarium)

“There’s something about the outside of a horse that is good for the inside of a man”, Churchill allegedly said. The horse’s motion has captured the interest of humans throughout history. Understanding of the mechanics of horse motion has been sought in early work by Aristotle (300 BC), in pioneering photographic studies by Muybridge (1880) as well as in modern day scientific publications.

The horse (Equus callabus ferus) is a remarkable animal athlete with outstanding running capabilities. The efficiency of its locomotion is explained by specialised anatomical features, which limit the degrees of freedom of movement and reduce energy consumption. Theoretical mechanical models are quite well suited to describe the essence of equine gaits and provide us with simple measures for analysing gait asymmetry. Such measures are well needed, since agreement between veterinarians is moderate to poor when it comes to visual assessment of lameness.

The human visual system has indeed clear limitations in perception and interpretation of horse motion. This limits our abilities to understand the horse, not only to detect lameness and to predict performance, but also to interpret its non-verbal communication and to detect signs of illness or discomfort.

This talk will provide a brief overview of existing motion analysis techniques and models in equine biomechanics. We will discuss future possibilities to achieve more accessible, sensitive and complex ways of analysing the motion of the horse.

  • Prof. Pietro Valdastri and Prof. Russel Harris
  • Room 2P4, Heisenbergstraße 3, 70569 Stuttgart

Prof. Pietro Valdastri's talk will focus on Medical Capsule Robots. Capsule robots are cm-size devices that leverage extreme miniaturization to access and operate in environments that are out of reach for larger robots. In medicine, capsule robots can be designed to be swallowed like a pill and to diagnose and treat mortal diseases, such as cancer. The talk will move from capsule robots for the inspection of the digestive tract toward a new generation of surgical robots and devices, having a relevant reduction in size, invasiveness, and cost as the main drivers for innovation. During the talk, we will discuss the recent enabling technologies that are being developed at the University of Leeds to transform medical robotics. These technologies include magnetic manipulation of capsule robots, hydraulic and pneumatic actuation, real-time tracking of capsule position and orientation, ultra-low-cost design, frugal innovation, and autonomy in robotic endoscopy.
Prof. Russell Harris has been researching new manufacturing processes for over 20 years. He has several research projects focussing on robotics, and is particularly interested in how new manufacturing processes can be an enabler to advanced robotic devices and components. In this talk he will discuss some of this research and where he believes there may be new opportunities for collaborative research across manufacturing and robotics.


  • Fabio Ferreira
  • aquarium

Endowing robots with human-like physical reasoning abilities remains challenging. We argue that existing methods often disregard spatio-temporal relations and by using Graph Neural Networks (GNNs) that incorporate a relational inductive bias, we can shift the learning process towards exploiting relations. In this work, we learn action-conditional forward dynamics models of a simulated manipulation task from visual observations involving cluttered and irregularly shaped objects. We investigate two GNN approaches and empirically assess their capability to generalize to scenarios with novel and an increasing number of objects. The first, Graph Networks (GN) based approach, considers explicitly defined edge attributes and not only does it consistently underperform an auto-encoder baseline that we modified to predict future states, our results indicate how different edge attributes can significantly influence the predictions. Consequently, we develop the Auto-Predictor that does not rely on explicitly defined edge attributes. It outperforms the baseline and the GN-based models. Overall, our results show the sensitivity of GNN-based approaches to the task representation, the efficacy of relational inductive biases and advocate choosing lightweight approaches that implicitly reason about relations over ones that leave these decisions to human designers.

Organizers: Siyu Tang


Soft Aerial Robotics for Infrastructure Manufacturing

Talk
  • 26 September 2019 • 14:00 15:00
  • Mirko Kovac
  • 2R04

Future cities and infrastructure systems will evolve into complex conglomerates where autonomous aerial, aquatic and ground-based robots will coexist with people and cooperate in symbiosis. To create this human-robot ecosystem, robots will need to respond more flexibly, robustly and efficiently than they do today. They will need to be designed with the ability to move across terrain boundaries and physically interact with infrastructure elements to perform sensing and intervention tasks. Taking inspiration from nature, aerial robotic systems can integrate multi-functional morphology, new materials, energy-efficient locomotion principles and advanced perception abilities that will allow them to successfully operate and cooperate in complex and dynamic environments. This talk will describe the scientific fundamentals, design principles and technologies for the development of biologically inspired flying robots with adaptive morphology that can perform monitoring and manufacturing tasks for future infrastructure and building systems. Examples will include flying robots with perching capabilities and origami-based landing systems, drones for aerial construction and repair, and combustion-based jet thrusters for aerial-aquatic vehicles.

Organizers: Metin Sitti


  • Eldar Insafutdinov
  • PS-Aquarium

In the first part of the talk, I am going to present our work on human pose estimation in the Wild, capturing unconstrained images and videos containing an a priori unknown number of people, often occluded and exhibiting a wide range of articulations and appearances. Unlike conventional top-down approaches that first detect humans with the off-the-shelf object detector and then estimate poses independently per bounding box, our formulation performs joint detection and pose estimation. In the first stage we indiscriminately localise body parts of every person in the image with the state-of-the-art ConvNet-based keypoint detector. In the second stage we perform assignment of keypoints to people based on a graph partitioning approach, that minimizes an integer linear program under a set of contraints and with the vertex and edge costs computed by our ConvNet. Our method naturally generalises to articulated tracking of multiple humans in video sequences. Next, I will discuss our work on learning accurate 3D object shape and camera pose from a collection of unlabeled category-specific images. We train a convolutional network to predict both the shape and the pose from a single image by minimizing the reprojection error: given several views of an object, the projections of the predicted shapes to the predicted camera poses should match the provided views. To deal with pose ambiguity, we introduce an ensemble of pose predictors that we then distill it to a single "student" model. To allow for efficient learning of high-fidelity shapes, we represent the shapes by point clouds and devise a formulation allowing for differentiable projection of these. Finally, I will talk about how to reconstruct an appearance of three-dimensional objects, namely a method for generating a 3D human avatar from an image. Our model predicts a full texture map, clothing segmentation and displacement map. The learning is done in the UV-space of the SMPL model, which turns the hard 3D inference problem into image-to-image translation task, where we can use deep neural networks to encode appearance, geometry and clothing layout. Our model is trained on a dataset of over 4000 3D scans of humans in diverse clothing.

Organizers: David Hoffmann Siyu Tang


From Fingertip Skin Mechanics to Dexterous Object Manipulation

IS Colloquium
  • 25 September 2019 • 13:00 14:00
  • Jean-Louis Thonnard
  • MPI-IS Stuttgart, Heisenbergstr. 3, Room 2P4

Fingertip skin friction plays a critical role during object manipulation. We will describe a simple and reliable method to estimate the fingertip static coefficient of friction (CF) continuously and quickly during object manipulation, and we will describe a global expression of the CF as a function of the normal force and fingertip moisture. Then we will show how skin hydration modifies the skin deformation dynamics during grip-like contacts. Certain motor behaviours observed during object manipulation could be explained by the effects of skin hydration. Then the biomechanics of the partial slip phenomenon will be described, and we will examine how this partial slip phenomenon is related to the subjective perception of fingertip slip.

Organizers: Katherine J. Kuchenbecker David Gueorguiev


  • Peter Blümler
  • MPI-IS Stuttgart, Heisenbergstr. 3, room 2P04

A new concept of using permanent magnet systems for guiding superparamagnetic nano-particles (SPP) on arbitrary trajectories over a large volume is presented. The same instrument can also be used for magnetic resonance imaging (MRI) using the inherent contrast of the SPP [1]. The basic idea is to use one magnet system, which provides a strong, homogeneous, dipolar magnetic field to magnetize and orient the particles, and a second constantly graded, quadrupolar field, superimposed on the first, to generate a force on the oriented particles. As a result, particles are guided with constant force and in a single direction over the entire volume. Prototypes of various sizes were constructed to demonstrate the principle in two dimensions on several nanoparticles, which were moved along a rough square by manual adjustment of the force angle [1]. Surprisingly even SPP with sizes < 100 nm could be moved with speeds exceeding 10 mm/s due to reversible agglomeration, for which a first hydrodynamic model is presented. Furthermore, a more advanced system with two quadrupoles is presented which allows canceling the force, hence stopping the SPP and moving them around sharp edges. Additionally, this system also allows for MRI and some first experiments are presented. Recently this concept was combined with liquid crystalline elastomers with incorporated SPP to create “micro-robots” whose coarse maneuvers are performed by a MagGuider-system while there microscopic actuation is controlled either by light or temperature [2]. 1. O. Baun, PB, JMMM 439 (2017) 294-304. doi: 10.1016/j.jmmm.2017.05.001 2. D. Ditter, PB et al. Adv. Functional Mater. 1902454 (2019) doi: 10.1002/adfm.201902454

Organizers: Metin Sitti


How to make your agent gesture in a natural way?

Talk
  • 19 September 2019 • 11:00 12:00
  • Taras Kucherenko
  • PS-Aquarium

Conversational agents in the form of virtual agents or social robots are rapidly becoming wide-spread. Humans use non-verbal behaviors to signal their intent, emotions and attitudes in human-human interactions. Conversational agents therefore need this ability as well in order to make an interaction pleasant and efficient. An important part of non-verbal communication is gesticulation: gestures communicate a large share of non-verbal content. Previous systems for gesture production were typically rule-based and could not represent the range of human gestures. Recently the gesture generation field has shifted to data-driven approaches. We follow this line of research by extending the state-of-the-art deep-learning based model. Our model leverages representation learning to enhance speech-gesture mapping. We provide analysis of different representations for the input (speech) and the output (motion) of the network by both objective and subjective evaluations. We also analyze the importance of smoothing of the produced motion and emphasize how challenging it is to evaluate gesture quality. In the future we plan to enrich input signal by taking semantic context (text transcription) as well, make the model probabilistic and evaluate our system on the social robot NAO.

Organizers: Mohamed Hassan Soubhik Sanyal


A Dynamical Systems Perspective on Optimization with Momentum

Talk
  • 19 September 2019 • 14:00 15:00
  • Dr. Michael Muehlebach
  • MPI-IS Stuttgart, Heisenbergstr. 3, seminar room 2P4

My talk will be divided into two parts. In the first part, I will analyze Nesterov's accelerated gradient method from a dynamical systems point of view. More precisely, I will derive the accelerated gradient method by discretizing an ordinary differential equation with a semi-implicit Euler integration scheme. I will analyze both the ordinary differential equation and the discretization for obtaining insights into the phenomenon of acceleration. In particular, geometric properties of the dynamics, such as asymptotic stability, time-reversibility, and phase-space volume contraction are shown to be preserved through the discretization. In the second part, I will show that these geometric properties are enough for characterizing the convergence rate. The results therefore provide criteria that are easily verifiable for the accelerated convergence of any momentum-based optimization algorithm. The results also yield guidance for the design of new optimization algorithms. The talk will focus on unconstrained optimization problems with smooth and strongly-convex objective functions, even though the analysis potentially generalizes to non-convex or non-Euclidean settings, or when the decision variables are constrained to a smooth manifold.

Organizers: Sebastian Trimpe


  • Ernest (Ted) Gomez, MD, MTR
  • MPI-IS Stuttgart, Heisenbergstr. 3, Room 2P4

Surgery is a demanding activity that places a human life in the hands of others. However, innovations in minimally invasive surgery have physically separated surgeons' hands from their patients, creating the need for surgeons and their tools to develop both natural and artificial haptic intelligence. This lecture examines the essential role of haptic intelligence in skill development for laparoscopic and robotic surgery.

Organizers: Katherine J. Kuchenbecker


Feedback-Control for Self-Adaptive Predictable Computing

Talk
  • 10 September 2019 • 14:00 15:00
  • Prof. Martina Maggio
  • MPI-IS Stuttgart, Heisenbergstr. 3, seminar room 2P4

Cloud computing gives the illusion of infinite computational capacity and allows for on-demand resource provisioning. As a result, over the last few years, the cloud computing model has experienced widespread industrial adoption and companies like Netflix offloaded their entire infrastructure to the cloud. However, with even the largest datacenter being of a finite size, cloud infrastructures have experienced overload due to overbooking or transient failures. In essence, this is an excellent opportunity for the design of control solutions, that tackle the problem of mitigating overload peaks, using feedback from the infrastructure. These solutions can then exploit control-theoretical principles and take advantage of the knowledge and the analysis capabilities of control tools to provide formal guarantees on the predictability of the infrastructure behavior. This talk introduces recent research advances on feedback control in the cloud computing domain, together with my research agenda for enhancing predictability and formal guarantees for cloud computing.

Organizers: Sebastian Trimpe