Publications | Max Planck Institute for Intelligent Systems

Ground reaction force sensing is one of the key components of gait analysis in legged locomotion research. To measure continuous force data during locomotion, we present a novel compound instrumented treadmill design. The treadmill is 1.7 m long, with a natural frequency of 170 Hz and an adjustable range that can be used for humans and small robots alike. Here, we present the treadmill’s design methodology and characterize it in its natural frequency, noise behavior and real-life performance. Additionally, we apply an ISO 376 norm conform calibration procedure for all spatial force directions and center of pressure position. We achieve a force accuracy of ≤ 5.6 N for the ground reaction forces and ≤ 13 mm in center of pressure position.

dlg

arXiv link (url) DOI [BibTex]

dlg Sarvestani, A., Ruppert, F., Badri-Spröwitz, A. An Open-Source Modular Treadmill for Dynamic Force Measurement with Load Dependant Range Adjustment 2023 (unpublished) Submitted

arXiv link (url) DOI [BibTex]

2022

Reconstructing Expressive 3D Humans from RGB Images

Choutas, V.

ETH Zurich, Max Planck Institute for Intelligent Systems and ETH Zurich, December 2022 (thesis)

Abstract

To interact with our environment, we need to adapt our body posture and grasp objects with our hands. During a conversation our facial expressions and hand gestures convey important non-verbal cues about our emotional state and intentions towards our fellow speakers. Thus, modeling and capturing 3D full-body shape and pose, hand articulation and facial expressions are necessary to create realistic human avatars for augmented and virtual reality. This is a complex task, due to the large number of degrees of freedom for articulation, body shape variance, occlusions from objects and self-occlusions from body parts, e.g. crossing our hands, and subject appearance. The community has thus far relied on expensive and cumbersome equipment, such as multi-view cameras or motion capture markers, to capture the 3D human body. While this approach is effective, it is limited to a small number of subjects and indoor scenarios. Using monocular RGB cameras would greatly simplify the avatar creation process, thanks to their lower cost and ease of use. These advantages come at a price though, since RGB capture methods need to deal with occlusions, perspective ambiguity and large variations in subject appearance, in addition to all the challenges posed by full-body capture. In an attempt to simplify the problem, researchers generally adopt a divide-and-conquer strategy, estimating the body, face and hands with distinct methods using part-specific datasets and benchmarks. However, the hands and face constrain the body and vice-versa, e.g. the position of the wrist depends on the elbow, shoulder, etc.; the divide-and-conquer approach can not utilize this constraint. In this thesis, we aim to reconstruct the full 3D human body, using only readily accessible monocular RGB images. In a first step, we introduce a parametric 3D body model, called SMPL-X, that can represent full-body shape and pose, hand articulation and facial expression. Next, we present an iterative optimization method, named SMPLify-X, that fits SMPL-X to 2D image keypoints. While SMPLify-X can produce plausible results if the 2D observations are sufficiently reliable, it is slow and susceptible to initialization. To overcome these limitations, we introduce ExPose, a neural network regressor, that predicts SMPL-X parameters from an image using body-driven attention, i.e. by zooming in on the hands and face, after predicting the body. From the zoomed-in part images, dedicated part networks predict the hand and face parameters. ExPose combines the independent body, hand, and face estimates by trusting them equally. This approach though does not fully exploit the correlation between parts and fails in the presence of challenges such as occlusion or motion blur. Thus, we need a better mechanism to aggregate information from the full body and part images. PIXIE uses neural networks called moderators that learn to fuse information from these two image sets before predicting the final part parameters. Overall, the addition of the hands and face leads to noticeably more natural and expressive reconstructions. Creating high fidelity avatars from RGB images requires accurate estimation of 3D body shape. Although existing methods are effective at predicting body pose, they struggle with body shape. We identify the lack of proper training data as the cause. To overcome this obstacle, we propose to collect internet images from fashion models websites, together with anthropometric measurements. At the same time, we ask human annotators to rate images and meshes according to a pre-defined set of linguistic attributes. We then define mappings between measurements, linguistic shape attributes and 3D body shape. Equipped with these mappings, we train a neural network regressor, SHAPY, that predicts accurate 3D body shapes from a single RGB image. We observe that existing 3D shape benchmarks lack subject variety and/or ground-truth shape. Thus, we introduce a new benchmark, Human Bodies in the Wild (HBW), which contains images of humans and their corresponding 3D ground-truth body shape. SHAPY shows how we can overcome the lack of in-the-wild images with 3D shape annotations through easy-to-obtain anthropometric measurements and linguistic shape attributes. Regressors that estimate 3D model parameters are robust and accurate, but often fail to tightly fit the observations. Optimization-based approaches tightly fit the data, by minimizing an energy function composed of a data term that penalizes deviations from the observations and priors that encode our knowledge of the problem. Finding the balance between these terms and implementing a performant version of the solver is a time-consuming and non-trivial task. Machine-learned continuous optimizers combine the benefits of both regression and optimization approaches. They learn the priors directly from data, avoiding the need for hand-crafted heuristics and loss term balancing, and benefit from optimized neural network frameworks for fast inference. Inspired from the classic Levenberg-Marquardt algorithm, we propose a neural optimizer that outperforms classic optimization, regression and hybrid optimization-regression approaches. Our proposed update rule uses a weighted combination of gradient descent and a network-predicted update. To show the versatility of the proposed method, we apply it on three other problems, namely full body estimation from (i) 2D keypoints, (ii) head and hand location from a head-mounted device and (iii) face tracking from dense 2D landmarks. Our method can easily be applied to new model fitting problems and offers a competitive alternative to well-tuned traditional model fitting pipelines, both in terms of accuracy and speed. To summarize, we propose a new and richer representation of the human body, SMPL-X, that is able to jointly model the 3D human body pose and shape, facial expressions and hand articulation. We propose methods, SMPLify-X, ExPose and PIXIE that estimate SMPL-X parameters from monocular RGB images, progressively improving the accuracy and realism of the predictions. To further improve reconstruction fidelity, we demonstrate how we can use easy-to-collect internet data and human annotations to overcome the lack of 3D shape data and train a model, SHAPY, that predicts accurate 3D body shape from a single RGB image. Finally, we propose a flexible learnable update rule for parametric human model fitting that outperforms both classic optimization and neural network approaches. This approach is easily applicable to a variety of problems, unlocking new applications in AR/VR scenarios.

pdf [BibTex]

2022

ps Choutas, V. Reconstructing Expressive 3D Humans from RGB Images ETH Zurich, Max Planck Institute for Intelligent Systems and ETH Zurich, December 2022 (thesis)

pdf [BibTex]

Investigating Independent Mechanisms in Neural Networks

Liang, W.

Université Paris-Saclay, France, October 2022 (mastersthesis)

[BibTex]

ei Liang, W. Investigating Independent Mechanisms in Neural Networks Université Paris-Saclay, France, October 2022 (mastersthesis)

[BibTex]

Modeling subgroup differences in fMRI data: disentangling subgroup-specific responses from shared ones

Keidar, D.

ETH Zurich, Switzerland, October 2022 (mastersthesis)

[BibTex]

ei Keidar, D. Modeling subgroup differences in fMRI data: disentangling subgroup-specific responses from shared ones ETH Zurich, Switzerland, October 2022 (mastersthesis)

[BibTex]

Multi-Target Multi-Object Manipulation using Relational Deep Reinforcement Learning

Feil, M.

Technnical University Munich, Germany, September 2022 (mastersthesis)

[BibTex]

ei Feil, M. Multi-Target Multi-Object Manipulation using Relational Deep Reinforcement Learning Technnical University Munich, Germany, September 2022 (mastersthesis)

[BibTex]

Independent Mechanism Analysis for High Dimensions

Sliwa, J.

University of Tübingen, Germany, September 2022, (Graduate Training Centre of Neuroscience) (mastersthesis)

[BibTex]

ei Sliwa, J. Independent Mechanism Analysis for High Dimensions University of Tübingen, Germany, September 2022, (Graduate Training Centre of Neuroscience) (mastersthesis)

[BibTex]

On the Adversarial Robustness of Causal Algorithmic Recourse

Dominguez-Olmedo, R.

University of Tübingen, Germany, August 2022 (mastersthesis)

[BibTex]

ei Dominguez-Olmedo, R. On the Adversarial Robustness of Causal Algorithmic Recourse University of Tübingen, Germany, August 2022 (mastersthesis)

[BibTex]

Independent Mechanism Analysis in High-Dimensional Observation Spaces

Ghosh, S.

ETH Zurich, Switzerland, June 2022 (mastersthesis)

[BibTex]

ei Ghosh, S. Independent Mechanism Analysis in High-Dimensional Observation Spaces ETH Zurich, Switzerland, June 2022 (mastersthesis)

[BibTex]

Voltage dependent investigations on the spin polarization of layered heterostructues

Miller, M.

Universität Stuttgart, Stuttgart, 2022 (mastersthesis)

mms

[BibTex]

mms Miller, M. Voltage dependent investigations on the spin polarization of layered heterostructues Universität Stuttgart, Stuttgart, 2022 (mastersthesis)

[BibTex]

2021

Learning Neural Causal Models with Active Interventions

Scherrer, N.

ETH Zurich, Switzerland, November 2021 (mastersthesis)

[BibTex]

2021

ei Scherrer, N. Learning Neural Causal Models with Active Interventions ETH Zurich, Switzerland, November 2021 (mastersthesis)

[BibTex]

HealthGen: Conditional Generation of Realistic Medical Time Series with Informative Missingness

Bing, S.

ETH Zurich, Switzerland, October 2021 (mastersthesis)

[BibTex]

ei Bing, S. HealthGen: Conditional Generation of Realistic Medical Time Series with Informative Missingness ETH Zurich, Switzerland, October 2021 (mastersthesis)

[BibTex]

Study of the Interventional Consistency of Autoencoders

Lanzillotta, G.

ETH Zurich, Switzerland, October 2021 (mastersthesis)

[BibTex]

ei Lanzillotta, G. Study of the Interventional Consistency of Autoencoders ETH Zurich, Switzerland, October 2021 (mastersthesis)

[BibTex]

Training with Few to Manipulate Many. On OOD generalization in relational reinforcement learning

Mambelli, D.

ETH Zurich, Switzerland, October 2021 (mastersthesis)

[BibTex]

ei Mambelli, D. Training with Few to Manipulate Many. On OOD generalization in relational reinforcement learning ETH Zurich, Switzerland, October 2021 (mastersthesis)

[BibTex]

Robotic Surgery Training in AR: Multimodal Record and Replay

Krauthausen, F.

pages: 1-147, University of Stuttgart, Stuttgart, May 2021, Study Program in Software Engineering (mastersthesis)

[BibTex]

hi Krauthausen, F. Robotic Surgery Training in AR: Multimodal Record and Replay pages: 1-147, University of Stuttgart, Stuttgart, May 2021, Study Program in Software Engineering (mastersthesis)

[BibTex]

Direct detection of spin Hall effect induced torques in platinum/ferromagnetic bilayer systems

Alten, F.

Universität Stuttgart, Stuttgart, January 2021 (mastersthesis)

mms

[BibTex]

mms Alten, F. Direct detection of spin Hall effect induced torques in platinum/ferromagnetic bilayer systems Universität Stuttgart, Stuttgart, January 2021 (mastersthesis)

[BibTex]

Community detection in heterogeneously attributed networks

Zottino, N.

Politecnico di Torino, 2021 (mastersthesis)

pio

link (url) [BibTex]

pio Zottino, N. Community detection in heterogeneously attributed networks Politecnico di Torino, 2021 (mastersthesis)

link (url) [BibTex]

2020

Voltage dependent interfacial magnetism in multilayer systems

Nacke, R.

Universität Stuttgart, Stuttgart, December 2020 (thesis)

mms

[BibTex]

2020

mms Nacke, R. Voltage dependent interfacial magnetism in multilayer systems Universität Stuttgart, Stuttgart, December 2020 (thesis)

[BibTex]

Hydromagnonics: Manipulation of magnonic systems with hydrogen

Sauter, R.

Universität Stuttgart, Stuttgart, December 2020 (mastersthesis)

mms

[BibTex]

mms Sauter, R. Hydromagnonics: Manipulation of magnonic systems with hydrogen Universität Stuttgart, Stuttgart, December 2020 (mastersthesis)

[BibTex]

A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

Ahmed, O.

ETH Zurich, Switzerland, October 2020 (mastersthesis)

[BibTex]

ei Ahmed, O. A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning ETH Zurich, Switzerland, October 2020 (mastersthesis)

[BibTex]

A Comprehensive Benchmark Evaluation of Synthetic Data Generation for Biomedical Imaging

DuMont Schütte, A.

ETH Zurich, Switzerland, October 2020 (mastersthesis)

[BibTex]

ei DuMont Schütte, A. A Comprehensive Benchmark Evaluation of Synthetic Data Generation for Biomedical Imaging ETH Zurich, Switzerland, October 2020 (mastersthesis)

[BibTex]

Deep learning for the parameter estimation of tight-binding Hamiltonians

Cacioppo, A.

University of Roma, La Sapienza, Italy, May 2020 (mastersthesis)

[BibTex]

ei Cacioppo, A. Deep learning for the parameter estimation of tight-binding Hamiltonians University of Roma, La Sapienza, Italy, May 2020 (mastersthesis)

[BibTex]

Learning Algorithms, Invariances, and the Real World

Zecevic, M.

Technical University of Darmstadt, Germany, April 2020 (mastersthesis)

[BibTex]

ei Zecevic, M. Learning Algorithms, Invariances, and the Real World Technical University of Darmstadt, Germany, April 2020 (mastersthesis)

[BibTex]

Interaction of hydrogen isotopes with flexible metal-organic frameworks

Bondorf, L.

Universität Stuttgart, Stuttgart, February 2020 (mastersthesis)

mms

[BibTex]

mms Bondorf, L. Interaction of hydrogen isotopes with flexible metal-organic frameworks Universität Stuttgart, Stuttgart, February 2020 (mastersthesis)

[BibTex]

Developing new methods for routing and optimal transport on networks

Lonardi, A.

Università degli studi di Padova, 2020 (mastersthesis)

pio

pdf [BibTex]

pio Lonardi, A. Developing new methods for routing and optimal transport on networks Università degli studi di Padova, 2020 (mastersthesis)

pdf [BibTex]

Edge-Disjoint Path Problem on Stochastic Block Models through Message Passing

Lorenzo Ferretti

Sapienza Università di Roma, 2020 (mastersthesis)

pio

[BibTex]

pio Lorenzo Ferretti Edge-Disjoint Path Problem on Stochastic Block Models through Message Passing Sapienza Università di Roma, 2020 (mastersthesis)

[BibTex]

Colloidal particles supporting urase activity

Baldauf, A.

Univ. of Stuttgart, 2020 (mastersthesis)

[BibTex]

pf Baldauf, A. Colloidal particles supporting urase activity Univ. of Stuttgart, 2020 (mastersthesis)

[BibTex]

Diffusion studies on biomolecules by NMR

Bochert, I.

Univ. of Stuttgart, 2020 (mastersthesis)

[BibTex]

pf Bochert, I. Diffusion studies on biomolecules by NMR Univ. of Stuttgart, 2020 (mastersthesis)

[BibTex]

2019

Analysis and modelling of information ecosystems

Emanuele Pigani

Università degli studi di Padova, October 2019 (mastersthesis)

pio

link (url) [BibTex]

2019

pio Emanuele Pigani Analysis and modelling of information ecosystems Università degli studi di Padova, October 2019 (mastersthesis)

link (url) [BibTex]

Inferring the Band Structure from Band Mapping Data through Machine Learning

Stimper, V.

Technical University of Munich, September 2019 (mastersthesis)

[BibTex]

ei Stimper, V. Inferring the Band Structure from Band Mapping Data through Machine Learning Technical University of Munich, September 2019 (mastersthesis)

[BibTex]

A new approach for community detection in multilayer networks

Contisciani, M.

Università degli studi di Padova, September 2019 (mastersthesis)

pio

link (url) [BibTex]

pio Contisciani, M. A new approach for community detection in multilayer networks Università degli studi di Padova, September 2019 (mastersthesis)

link (url) [BibTex]

Learning to Diagnose Diabetes from Magnetic Resonance Tomography

Dietz, B.

ETH Zurich, Switzerland, August 2019 (mastersthesis)

[BibTex]

ei Dietz, B. Learning to Diagnose Diabetes from Magnetic Resonance Tomography ETH Zurich, Switzerland, August 2019 (mastersthesis)

[BibTex]

Reinforcement Learning for a Two-Robot Table Tennis Simulation

Li, G.

RWTH Aachen University, Germany, July 2019 (mastersthesis)

[BibTex]

ei Li, G. Reinforcement Learning for a Two-Robot Table Tennis Simulation RWTH Aachen University, Germany, July 2019 (mastersthesis)

[BibTex]

Characteristics of longitudinal physiological measurements of late-stage ALS patients

Konieczny, L.

Ludwig-Maximilians-Universität München, Germany, May 2019 (mastersthesis)

[BibTex]

ei Konieczny, L. Characteristics of longitudinal physiological measurements of late-stage ALS patients Ludwig-Maximilians-Universität München, Germany, May 2019 (mastersthesis)

[BibTex]

X-ray microscopic characterization of high-Tc-supercoductors using image processing

Bihler, M.

Universität Stuttgart, Stuttgart, 2019 (mastersthesis)

mms

[BibTex]

mms Bihler, M. X-ray microscopic characterization of high-Tc-supercoductors using image processing Universität Stuttgart, Stuttgart, 2019 (mastersthesis)

[BibTex]

Active matter and self propelled microparticles

Kottapalli, S. N. M.

Univ. of Stuttgart, 2019 (mastersthesis)

[BibTex]

pf Kottapalli, S. N. M. Active matter and self propelled microparticles Univ. of Stuttgart, 2019 (mastersthesis)

[BibTex]

Dynamic hierarchical ranking in directed networks: from static SpringRank to dynamic and bayesian models using variational techniques

Vecchia, A. D.

Università degli studi di Padova, 2019 (mastersthesis)

pio

pdf [BibTex]

pio Vecchia, A. D. Dynamic hierarchical ranking in directed networks: from static SpringRank to dynamic and bayesian models using variational techniques Università degli studi di Padova, 2019 (mastersthesis)

pdf [BibTex]

Spatial Filtering based on Riemannian Manifold for Brain-Computer Interfacing

Xu, J.

Technical University of Munich, Germany, 2019 (mastersthesis)

[BibTex]

ei Xu, J. Spatial Filtering based on Riemannian Manifold for Brain-Computer Interfacing Technical University of Munich, Germany, 2019 (mastersthesis)

[BibTex]

Electronics, Software and Analysis of a Bioinspired Sensorized Quadrupedal Robot

Petereit, R.

Technische Universität München, 2019 (mastersthesis)

dlg

[BibTex]

dlg Petereit, R. Electronics, Software and Analysis of a Bioinspired Sensorized Quadrupedal Robot Technische Universität München, 2019 (mastersthesis)

[BibTex]

Haptic Reality: Novel Interfacing for Informed Assembly Systems

Tashiro, N., Faulkner, R., Melnyk, S., Rosales, T.

University of Stuttgart, 2019 (mastersthesis)

[BibTex]

hi Tashiro, N., Faulkner, R., Melnyk, S., Rosales, T. Haptic Reality: Novel Interfacing for Informed Assembly Systems University of Stuttgart, 2019 (mastersthesis)

[BibTex]

Variational Inference: Recent Advances and Applications to Dynamic Ranking on Graphs

Ruggeri, N.

Università degli studi di Padova, 2019 (mastersthesis)

pio

pdf [BibTex]

pio Ruggeri, N. Variational Inference: Recent Advances and Applications to Dynamic Ranking on Graphs Università degli studi di Padova, 2019 (mastersthesis)

pdf [BibTex]

Entwicklung und Analyse neuartiger fluidischer Aktoren mit Rollmembran

Hepp, J.

Technische Universität München, 2019 (mastersthesis)

dlg

Project Page [BibTex]

dlg Hepp, J. Entwicklung und Analyse neuartiger fluidischer Aktoren mit Rollmembran Technische Universität München, 2019 (mastersthesis)

Project Page [BibTex]

2018

Gait analysis of running guinea fowls

Bonnet, A.

August 2018 (mastersthesis)

dlg

[BibTex]

2018

dlg Bonnet, A. Gait analysis of running guinea fowls August 2018 (mastersthesis)

[BibTex]

Robust Visual Augmented Reality in Robot-Assisted Surgery

Forte, M.

Politecnico di Milano, Milan, Italy, July 2018, Department of Electronic, Information, and Biomedical Engineering (mastersthesis)

Abstract

The broader research objective of this line of research is to test the hypothesis that real-time stereo video analysis and augmented reality can increase safety and task efficiency in robot-assisted surgery. This master’s thesis aims to solve the first step needed to achieve this goal: the creation of a robust system that delivers the envisioned feedback to a surgeon while he or she controls a surgical robot that is identical to those used on human patients. Several approaches for applying augmented reality to da Vinci Surgical Systems have been proposed, but none of them entirely rely on a clinical robot; specifically, they require additional sensors, depend on access to the da Vinci API, are designed for a very specific task, or were tested on systems that are starkly different from those in clinical use. There has also been prior work that presents the real-world camera view and the computer graphics on separate screens, or not in real time. In other scenarios, the digital information is overlaid manually by the surgeons themselves or by computer scientists, rather than being generated automatically in response to the surgeon’s actions. We attempted to overcome the aforementioned constraints by acquiring input signals from the da Vinci stereo endoscope and providing augmented reality to the console in real time (less than 150 ms delay, including the 62 ms of inherent latency of the da Vinci). The potential benefits of the resulting system are broad because it was built to be general, rather than customized for any specific task. The entire platform is compatible with any generation of the da Vinci System and does not require a dVRK (da Vinci Research Kit) or access to the API. Thus, it can be applied to existing da Vinci Systems in operating rooms around the world.

[BibTex]

hi Forte, M. Robust Visual Augmented Reality in Robot-Assisted Surgery Politecnico di Milano, Milan, Italy, July 2018, Department of Electronic, Information, and Biomedical Engineering (mastersthesis)

[BibTex]