Applying data-driven approaches to non-rigid 3D reconstruction has been difficult, which we believe can be attributed to the lack of a large-scale training corpus. One recent approach proposes self-supervision based on non-rigid reconstruction. Unfortunately, this method fails for important cases such as highly non-rigid deformations. We first address this problem of lack of data by introducing a novel semi-supervised strategy to obtain dense interframe correspondences from a sparse set of annotations. This way, we obtain a large dataset of 400 scenes, over 390,000 RGB-D frames, and 2,537 densely aligned frame pairs; in addition, we provide a test set along with several metrics for evaluation. Based on this corpus, we introduce a data-driven non-rigid feature matching approach, which we integrate into an optimization-based reconstruction pipeline. Here, we propose a new neural network that operates on RGB-D frames, while maintaining robustness under large non-rigid deformations and producing accurate predictions. Our approach significantly outperforms both existing non-rigid reconstruction methods that do not use learned data terms, as well as learning-based approaches that only use self-supervision.
Organizers: Vassilis Choutas
How can we tell that a video is playing backwards? People's motions look wrong when the video is played backwards--can we develop an algorithm to distinguish forward from backward video? Similarly, can we tell if a video is sped-up? We have developed algorithms to distinguish forwards from backwards video, and fast from slow. Training algorithms for these tasks provides a self-supervised task that facilitates human activity recognition. We'll show these results, and applications of these unsupervised video learning tasks. We also present a method to retime people in videos --- manipulating and editing the time over which the motions of individuals occurs. Our model not only disentangles the motions of each person in the video, but it also correlates each person with the scene changes they generate, and thus re-times the corresponding shadows, reflections, and motion of loose clothing appropriately.
Organizers: Yinghao Huang
In recent years, commodity 3D sensors have become widely available, spawning significant interest in both offline and real-time 3D reconstruction. While state-of-the-art reconstruction results from commodity RGB-D sensors are visually appealing, they are far from usable in practical computer graphics applications since they do not match the high quality of artist-modeled 3D graphics content. One of the biggest challenges in this context is that obtained 3D scans suffer from occlusions, thus resulting in incomplete 3D models. In this talk, I will present a data-driven approach towards generating high quality 3D models from commodity scan data, and the use of these geometrically complete 3D models towards semantic and texture understanding of real-world environments.
Organizers: Yinghao Huang
Security and privacy is of growing concern in many control applications. Cyber attacks are frequently reported for a variety of industrial and infrastructure systems. For more than a decade the control community has developed techniques for how to design control systems resilient to cyber-physical attacks. In this talk, we will review some of these results. In particular, as cyber and physical components of networked control systems are tightly interconnected, it is be argued that traditional IT security focusing only on the cyber part does not provide appropriate solutions. Modeling the objectives and resources of the adversary together with the plant and control dynamics is shown to be essential. The consequences of common attack scenarios, such denial-of-service, replay, and bias injection attacks, can be analyzed using the presented framework. It is also shown how to strengthen the control loops by deriving security and privacy aware estimation and control schemes. Applications in building automation, power networks, and automotive systems will be used to motivate and illustrate the results. The presentation is based on joint work with several students and colleagues at KTH and elsewhere.
In the search for materials with new properties, there have been great advances in recent years aimed at the construction of mechanical systems whose behaviour is governed by structure, rather than composition. Through careful design of the material’s architecture, new material properties have been demonstrated, including negative Poisson’s ratio, high stiffness-to-weight ratio and mechanical cloaking. While originally the field focused on achieving unusual (zero or negative) values for familiar mechanical parameters, more recently it has been shown that non-linearities can be exploited to further extend the design space. In this talk Prof. Katia Bertoldi will focus on kirigami-inspired metamaterials, which are produced by introducing arrays of cuts into thin sheets. First, she will demonstrate that instabilities triggered under uniaxial tension can be exploited to create complex 3D patterns and even to guide the formation of permanent folds. Second, she will show that such non-linear systems can be used to designs smart and flexible skins with anisotropic frictional properties that enables a single soft actuator to propel itself. Finally, Prof.Bertoldi will focus on bistable kirigami metamaterials and show that they provide an ideal environment for the propagation non-linear waves.
Organizers: Metin Sitti
Machine learning increasingly supports consequential decisions in domains including health, employment, and criminal justice. Consequential decision making is inherently dynamic: Individuals, their outcomes, and entire populations can change and adapt in response to classification. Traditional machine learning, however, fails to account for such dynamic effects. In this talk, I will highlight three different vignettes of dynamic decision making. The first is about how classification changes populations and how this perspective is essential to questions of fairness in machine learning. The second is about how classification incentivizes individuals to adapt strategically. The third is about how predictions are often performative, that is, they influence the very outcome they aim to predict. I will end on the contours of a theory that unifies these three settings and its connections to questions in causality, control theory, economics, and sociology.
Organizers: Metin Sitti
Recent economic, technological, and societal changes (e.g., the shift from large organizations to decentralized networks of individuals/small businesses, #metoo movement) require organizations to adapt to the transforming nature of work by altering the way work is performed and the roles that workers play. Due to globalization and advanced communication technologies, modern organizations are also characterized by a diverse workforce that needs to be carefully managed. Therefore organizational leaders must take on the challenge of unleashing the true potential of diversity and inclusion by challenging assumptions and changing corporate cultures. Using qualitative and quantitative methods, my research explores the implications of the ongoing transformation of work in terms of worker’s identity and their perspective on time and place, and looks into the main competencies workers need to successfully adapt to the new way of working. In addition, I examine how modern organizations can promote a diverse and inclusive workplace and how they deal with one the major barriers to the career development of professional women – sexual harassment in the workplace. Finally, I explore the role of leaders in creating a diverse and inclusive community. On a larger scale, my research aims to help leaders and organizations clarify how they can contribute to a more tolerant, diverse, and inclusive society.
Organizers: Katherine J. Kuchenbecker
The demand for safe, robust, and intelligent robotic systems is growing rapidly, given their potential to make our societies more productive and increase our welfare. To achieve this, robots are increasingly expected to operate in human-populated environments, maneuver in remote and cluttered environments, maintain and repair facilities, take care of our health, and streamline manufacturing and assembly lines. However, computational issues limit the ability of robots to plan complex motions in constrained and contact-rich environments, interact with humans safely, and exploit dynamics to gracefully maneuver, manipulate, fly, or explore the oceans. This talk will be centered around planning and decision-making algorithms for robust and agile robots operating in complex environments. In particular, Dr. Zhao will present novel computational approaches necessary to enable real-time and robust motion planning of highly dynamic bipedal locomotion over rough terrain. This planning approach revolves around robust disturbance metrics, an optimal recovery controller, and foot placement re-planning strategies. Extending this motion planning approach to generalized whole-body locomotion behaviors, He will introduce our recent progress on high-level reactive task planner synthesis for multi-contact, template-based locomotion interacting with constrained environments and how to integrate formal methods for mission-capable locomotion. This talk will also present robust trajectory optimization algorithm capable of handling contact uncertainties and without enumerating contact modes. Dr. Zhao will end this talk with current research directions on distributed trajectory optimization and task and motion planning.
Organizers: Metin Sitti
Fernanda Bribiesca-Contreras' doctoral research investigates the form and function relationship of the wing muscles and its implication with aerial and underwater flight.
Organizers: Alexander Badri-Sprowitz
Our scientific understanding of haptic interaction is still evolving, both because what you feel greatly depends on how you move, and because engineered sensors, actuators, and algorithms typically struggle to match human capabilities. Consequently, few computer and machine interfaces provide the human operator with high-fidelity touch feedback or carefully analyze the physical signals generated during haptic interactions, limiting their usability. The crucial role of the sense of touch is also deeply appreciated by researchers working to create autonomous robots that can competently manipulate everyday objects and safely interact with humans in unstructured environments.
Providing rich and immersive physical experiences to users has become an essential component in many computer-interactive applications, where haptics plays a central role. However, as with other sensory modalities, modeling and rendering good haptic experiences with plausible physicality is a very demanding task in terms of the cost associated with modeling and authoring, not to mention the cost for development. No general and widely-used solutions exist yet for that; most designers and developers rely on their in-house programs, or even worse, manual coding. This talk will introduce the research conducted by the speaker in order to facilitate the authoring of haptic content. In particular, it will focus on automatic synthesis algorithms of vibrotactile effects and motion effects from audiovisual content, as well as some relevant issues in haptic perception.
Organizers: Katherine J. Kuchenbecker
In this talk, I will present about the most recent advances in data-driven character animation and control using neural networks. Creating key-framed animations by hand is typically very time-consuming and requires a lot of artistic expertise and training. Recent work applying deep learning for character animation was firstly able to compete or even outperform the quality that could be achieved by professional animators for biped locomotion, and thus caused a lot excitement in both academia and industry. Shortly after, following research also demonstrated its applicability to quadruped locomotion control, which has been considered one of the unsolved key challenges in character animation due to the highly complex footfall patterns of quadruped characters. Addressing the next challenges beyond character locomotion, this year at SIGGRAPH Asia we presented the Neural State Machine, an improved version of such previous systems in order to make human characters naturally interact with objects and the environment from motion capture data. Generally, the difficulty in such tasks is due to complex planning of periodic and aperiodic movements reacting to the scene geometry in order to precisely position and orient the character, and to adapt to different variations in the type, size and shape of such objects. We demonstrate the versatility of this framework with various scene interaction tasks, such as sitting on a chair, avoiding obstacles, opening and entering through a door, and picking and carrying objects generated in real-time just from a single model.
The body is one of the most relevant aspects of our self, and we shape it through our eating behavior and physical acitivity. As a psychologist and neuroscientist, I seek to disentangle mutual interactions between how we represent our own body, what we eat and how much we exercise. In the talk, I will give a scoping overview of this approach and present the studies I am conducting as a guest scientist at PS.
Organizers: Ahmed Osman