Imagine a futuristic version of Google Street View that could dial up any possible place in the world, at any possible time. Effectively, such a service would be a recording of the plenoptic function—the hypothetical function described by Adelson and Bergen that captures all light rays passing through space at all times. While the plenoptic function is completely impractical to capture in its totality, every photo ever taken represents a sample of this function. I will present recent methods we've developed to reconstruct the plenoptic function from sparse space-time samples of photos—including Street View itself, as well as tourist photos of famous landmarks. The results of this work include the ability to take a single photo and synthesize a full dawn-to-dusk timelapse video, as well as compelling 4D view synthesis capabilities where a scene can simultaneously be explored in space and time.
One of the most striking characteristics of human behavior in contrast to all other animal is that we show extraordinary variability across populations. Human cultural diversity is a biological oddity. More specifically, we propose that what makes humans unique is the nature of the individual ontogenetic process, that results in this unparalleled cultural diversity. Hence, our central question is: How is human ontogeny adapted to cultural diversity and how does it contribute to it? This question is critical, because cultural diversity does not only entail our predominant mode of adaptation to local ecologies, but is key in the construction of our cognitive architecture. The colors we see, the tones that we hear, the memories we form, the norms we adhere to are all the consequence of an interaction between our emerging cognitive system and our lived experiences. While psychologists make careers measuring cognitive systems, we are terrible at measuring experience as are anthropologists, sociologists, etc. The standard methods all face unsurmountable limitations. In our department, we hope to apply Machine Learning, Deep Learning and Computer Vision to automatically extract developmentally important indicators of humans’ daily experience. Similarly to the way that modern sequencing technologies allow us to study the human genotype at scale, applying AI methods to reliably quantify humans’ lived experience would allow us to study the human behavioral phenotype at scale, and fundamentally alter the science of human behavior and its application in education, mental health and medicine: The phenotyping revolution.
Organizers: Timo Bolkart
Prof. Eric Dufresne will describe some experiments on some simple composites of elastomers and droplets. First, we will consider their composite mechanical properties. He will show how simple liquid droplets can counterintuitively stiffen the material, and how magnetorheological fluid droplets can provide elastomers with magnetically switchable shape memory. Second, we consider the nucleation, growth, and ripening of droplets within an elastomer. Here, a variety of interesting phenomena emerge: size-tunable monodisperse droplets, shape-tunable droplets, and ripening of droplets along stiffness gradients. We are exploiting these phenomena to make materials with mechanically switchable structural color.
Organizers: Metin Sitti
Computation has fundamentally changed the way we study nature. New data collection technology, such as GPS, high definition cameras, UAVs, genotyping, and crowdsourcing, are generating data about wild populations that are orders of magnitude richer than any previously collected. Unfortunately, in this domain as in many others, our ability to analyze data lags substantially behind our ability to collect it. In this talk I will show how computational approaches can be part of every stage of the scientific process of understanding animal sociality, from intelligent data collection (crowdsourcing photographs and identifying individual animals from photographs by stripes and spots - Wildbook.org) to hypothesis formulation (by designing a novel computational framework for analysis of dynamic social networks), and provide scientific insight into collective behavior of zebras, baboons, and other social animals.
Organizers: Aamir Ahmad
Prof. Pietro Valdastri's talk will focus on Medical Capsule Robots. Capsule robots are cm-size devices that leverage extreme miniaturization to access and operate in environments that are out of reach for larger robots. In medicine, capsule robots can be designed to be swallowed like a pill and to diagnose and treat mortal diseases, such as cancer. The talk will move from capsule robots for the inspection of the digestive tract toward a new generation of surgical robots and devices, having a relevant reduction in size, invasiveness, and cost as the main drivers for innovation. During the talk, we will discuss the recent enabling technologies that are being developed at the University of Leeds to transform medical robotics. These technologies include magnetic manipulation of capsule robots, hydraulic and pneumatic actuation, real-time tracking of capsule position and orientation, ultra-low-cost design, frugal innovation, and autonomy in robotic endoscopy. Prof. Russell Harris has been researching new manufacturing processes for over 20 years. He has several research projects focussing on robotics, and is particularly interested in how new manufacturing processes can be an enabler to advanced robotic devices and components. In this talk he will discuss some of this research and where he believes there may be new opportunities for collaborative research across manufacturing and robotics.
Endowing robots with human-like physical reasoning abilities remains challenging. We argue that existing methods often disregard spatio-temporal relations and by using Graph Neural Networks (GNNs) that incorporate a relational inductive bias, we can shift the learning process towards exploiting relations. In this work, we learn action-conditional forward dynamics models of a simulated manipulation task from visual observations involving cluttered and irregularly shaped objects. We investigate two GNN approaches and empirically assess their capability to generalize to scenarios with novel and an increasing number of objects. The first, Graph Networks (GN) based approach, considers explicitly defined edge attributes and not only does it consistently underperform an auto-encoder baseline that we modified to predict future states, our results indicate how different edge attributes can significantly influence the predictions. Consequently, we develop the Auto-Predictor that does not rely on explicitly defined edge attributes. It outperforms the baseline and the GN-based models. Overall, our results show the sensitivity of GNN-based approaches to the task representation, the efficacy of relational inductive biases and advocate choosing lightweight approaches that implicitly reason about relations over ones that leave these decisions to human designers.
Organizers: Siyu Tang
Future cities and infrastructure systems will evolve into complex conglomerates where autonomous aerial, aquatic and ground-based robots will coexist with people and cooperate in symbiosis. To create this human-robot ecosystem, robots will need to respond more flexibly, robustly and efficiently than they do today. They will need to be designed with the ability to move across terrain boundaries and physically interact with infrastructure elements to perform sensing and intervention tasks. Taking inspiration from nature, aerial robotic systems can integrate multi-functional morphology, new materials, energy-efficient locomotion principles and advanced perception abilities that will allow them to successfully operate and cooperate in complex and dynamic environments. This talk will describe the scientific fundamentals, design principles and technologies for the development of biologically inspired flying robots with adaptive morphology that can perform monitoring and manufacturing tasks for future infrastructure and building systems. Examples will include flying robots with perching capabilities and origami-based landing systems, drones for aerial construction and repair, and combustion-based jet thrusters for aerial-aquatic vehicles.
Organizers: Metin Sitti
In the first part of the talk, I am going to present our work on human pose estimation in the Wild, capturing unconstrained images and videos containing an a priori unknown number of people, often occluded and exhibiting a wide range of articulations and appearances. Unlike conventional top-down approaches that first detect humans with the off-the-shelf object detector and then estimate poses independently per bounding box, our formulation performs joint detection and pose estimation. In the first stage we indiscriminately localise body parts of every person in the image with the state-of-the-art ConvNet-based keypoint detector. In the second stage we perform assignment of keypoints to people based on a graph partitioning approach, that minimizes an integer linear program under a set of contraints and with the vertex and edge costs computed by our ConvNet. Our method naturally generalises to articulated tracking of multiple humans in video sequences. Next, I will discuss our work on learning accurate 3D object shape and camera pose from a collection of unlabeled category-specific images. We train a convolutional network to predict both the shape and the pose from a single image by minimizing the reprojection error: given several views of an object, the projections of the predicted shapes to the predicted camera poses should match the provided views. To deal with pose ambiguity, we introduce an ensemble of pose predictors that we then distill it to a single "student" model. To allow for efficient learning of high-fidelity shapes, we represent the shapes by point clouds and devise a formulation allowing for differentiable projection of these. Finally, I will talk about how to reconstruct an appearance of three-dimensional objects, namely a method for generating a 3D human avatar from an image. Our model predicts a full texture map, clothing segmentation and displacement map. The learning is done in the UV-space of the SMPL model, which turns the hard 3D inference problem into image-to-image translation task, where we can use deep neural networks to encode appearance, geometry and clothing layout. Our model is trained on a dataset of over 4000 3D scans of humans in diverse clothing.
Fingertip skin friction plays a critical role during object manipulation. We will describe a simple and reliable method to estimate the fingertip static coefficient of friction (CF) continuously and quickly during object manipulation, and we will describe a global expression of the CF as a function of the normal force and fingertip moisture. Then we will show how skin hydration modifies the skin deformation dynamics during grip-like contacts. Certain motor behaviours observed during object manipulation could be explained by the effects of skin hydration. Then the biomechanics of the partial slip phenomenon will be described, and we will examine how this partial slip phenomenon is related to the subjective perception of fingertip slip.
Organizers: Katherine J. Kuchenbecker
A new concept of using permanent magnet systems for guiding superparamagnetic nano-particles (SPP) on arbitrary trajectories over a large volume is presented. The same instrument can also be used for magnetic resonance imaging (MRI) using the inherent contrast of the SPP . The basic idea is to use one magnet system, which provides a strong, homogeneous, dipolar magnetic field to magnetize and orient the particles, and a second constantly graded, quadrupolar field, superimposed on the first, to generate a force on the oriented particles. As a result, particles are guided with constant force and in a single direction over the entire volume. Prototypes of various sizes were constructed to demonstrate the principle in two dimensions on several nanoparticles, which were moved along a rough square by manual adjustment of the force angle . Surprisingly even SPP with sizes < 100 nm could be moved with speeds exceeding 10 mm/s due to reversible agglomeration, for which a first hydrodynamic model is presented. Furthermore, a more advanced system with two quadrupoles is presented which allows canceling the force, hence stopping the SPP and moving them around sharp edges. Additionally, this system also allows for MRI and some first experiments are presented. Recently this concept was combined with liquid crystalline elastomers with incorporated SPP to create “micro-robots” whose coarse maneuvers are performed by a MagGuider-system while there microscopic actuation is controlled either by light or temperature . 1. O. Baun, PB, JMMM 439 (2017) 294-304. doi: 10.1016/j.jmmm.2017.05.001 2. D. Ditter, PB et al. Adv. Functional Mater. 1902454 (2019) doi: 10.1002/adfm.201902454
Organizers: Metin Sitti
Conversational agents in the form of virtual agents or social robots are rapidly becoming wide-spread. Humans use non-verbal behaviors to signal their intent, emotions and attitudes in human-human interactions. Conversational agents therefore need this ability as well in order to make an interaction pleasant and efficient. An important part of non-verbal communication is gesticulation: gestures communicate a large share of non-verbal content. Previous systems for gesture production were typically rule-based and could not represent the range of human gestures. Recently the gesture generation field has shifted to data-driven approaches. We follow this line of research by extending the state-of-the-art deep-learning based model. Our model leverages representation learning to enhance speech-gesture mapping. We provide analysis of different representations for the input (speech) and the output (motion) of the network by both objective and subjective evaluations. We also analyze the importance of smoothing of the produced motion and emphasize how challenging it is to evaluate gesture quality. In the future we plan to enrich input signal by taking semantic context (text transcription) as well, make the model probabilistic and evaluate our system on the social robot NAO.
My talk will be divided into two parts. In the first part, I will analyze Nesterov's accelerated gradient method from a dynamical systems point of view. More precisely, I will derive the accelerated gradient method by discretizing an ordinary differential equation with a semi-implicit Euler integration scheme. I will analyze both the ordinary differential equation and the discretization for obtaining insights into the phenomenon of acceleration. In particular, geometric properties of the dynamics, such as asymptotic stability, time-reversibility, and phase-space volume contraction are shown to be preserved through the discretization. In the second part, I will show that these geometric properties are enough for characterizing the convergence rate. The results therefore provide criteria that are easily verifiable for the accelerated convergence of any momentum-based optimization algorithm. The results also yield guidance for the design of new optimization algorithms. The talk will focus on unconstrained optimization problems with smooth and strongly-convex objective functions, even though the analysis potentially generalizes to non-convex or non-Euclidean settings, or when the decision variables are constrained to a smooth manifold.
Organizers: Sebastian Trimpe