Header logo is


2013


Probabilistic Object Tracking Using a Range Camera
Probabilistic Object Tracking Using a Range Camera

Wüthrich, M., Pastor, P., Kalakrishnan, M., Bohg, J., Schaal, S.

In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages: 3195-3202, IEEE, November 2013 (inproceedings)

Abstract
We address the problem of tracking the 6-DoF pose of an object while it is being manipulated by a human or a robot. We use a dynamic Bayesian network to perform inference and compute a posterior distribution over the current object pose. Depending on whether a robot or a human manipulates the object, we employ a process model with or without knowledge of control inputs. Observations are obtained from a range camera. As opposed to previous object tracking methods, we explicitly model self-occlusions and occlusions from the environment, e.g, the human or robotic hand. This leads to a strongly non-linear observation model and additional dependencies in the Bayesian network. We employ a Rao-Blackwellised particle filter to compute an estimate of the object pose at every time step. In a set of experiments, we demonstrate the ability of our method to accurately and robustly track the object pose in real-time while it is being manipulated by a human or a robot.

am

arXiv Video Code Video DOI Project Page [BibTex]

2013


arXiv Video Code Video DOI Project Page [BibTex]


3-D Object Reconstruction of Symmetric Objects by Fusing Visual and Tactile Sensing
3-D Object Reconstruction of Symmetric Objects by Fusing Visual and Tactile Sensing

Illonen, J., Bohg, J., Kyrki, V.

The International Journal of Robotics Research, 33(2):321-341, Sage, October 2013 (article)

Abstract
In this work, we propose to reconstruct a complete 3-D model of an unknown object by fusion of visual and tactile information while the object is grasped. Assuming the object is symmetric, a first hypothesis of its complete 3-D shape is generated. A grasp is executed on the object with a robotic manipulator equipped with tactile sensors. Given the detected contacts between the fingers and the object, the initial full object model including the symmetry parameters can be refined. This refined model will then allow the planning of more complex manipulation tasks. The main contribution of this work is an optimal estimation approach for the fusion of visual and tactile data applying the constraint of object symmetry. The fusion is formulated as a state estimation problem and solved with an iterative extended Kalman filter. The approach is validated experimentally using both artificial and real data from two different robotic platforms.

am

Web DOI Project Page [BibTex]

Web DOI Project Page [BibTex]


no image
Dry adhesives and methods for making dry adhesives

Sitti, M., Kim, S.

sep 2013, US Patent App. 14/016,651 (misc)

pi

[BibTex]

[BibTex]


no image
Dry adhesives and methods for making dry adhesives

Sitti, M., Kim, S.

sep 2013, US Patent App. 14/016,683 (misc)

pi

[BibTex]

[BibTex]


no image
Dry adhesives and methods for making dry adhesives

Sitti, M., Kim, S.

sep 2013, US Patent 8,524,092 (misc)

pi

[BibTex]

[BibTex]


Learning and Optimization with Submodular Functions
Learning and Optimization with Submodular Functions

Sankaran, B., Ghazvininejad, M., He, X., Kale, D., Cohen, L.

ArXiv, May 2013 (techreport)

Abstract
In many naturally occurring optimization problems one needs to ensure that the definition of the optimization problem lends itself to solutions that are tractable to compute. In cases where exact solutions cannot be computed tractably, it is beneficial to have strong guarantees on the tractable approximate solutions. In order operate under these criterion most optimization problems are cast under the umbrella of convexity or submodularity. In this report we will study design and optimization over a common class of functions called submodular functions. Set functions, and specifically submodular set functions, characterize a wide variety of naturally occurring optimization problems, and the property of submodularity of set functions has deep theoretical consequences with wide ranging applications. Informally, the property of submodularity of set functions concerns the intuitive principle of diminishing returns. This property states that adding an element to a smaller set has more value than adding it to a larger set. Common examples of submodular monotone functions are entropies, concave functions of cardinality, and matroid rank functions; non-monotone examples include graph cuts, network flows, and mutual information. In this paper we will review the formal definition of submodularity; the optimization of submodular functions, both maximization and minimization; and finally discuss some applications in relation to learning and reasoning using submodular functions.

am

arxiv link (url) [BibTex]

arxiv link (url) [BibTex]


Hypothesis Testing Framework for Active Object Detection
Hypothesis Testing Framework for Active Object Detection

Sankaran, B., Atanasov, N., Le Ny, J., Koletschka, T., Pappas, G., Daniilidis, K.

In IEEE International Conference on Robotics and Automation (ICRA), May 2013, clmc (inproceedings)

Abstract
One of the central problems in computer vision is the detection of semantically important objects and the estimation of their pose. Most of the work in object detection has been based on single image processing and its performance is limited by occlusions and ambiguity in appearance and geometry. This paper proposes an active approach to object detection by controlling the point of view of a mobile depth camera. When an initial static detection phase identifies an object of interest, several hypotheses are made about its class and orientation. The sensor then plans a sequence of view-points, which balances the amount of energy used to move with the chance of identifying the correct hypothesis. We formulate an active M-ary hypothesis testing problem, which includes sensor mobility, and solve it using a point-based approximate POMDP algorithm. The validity of our approach is verified through simulation and experiments with real scenes captured by a kinect sensor. The results suggest a significant improvement over static object detection.

am

pdf [BibTex]

pdf [BibTex]


no image
Dry adhesives and methods of making dry adhesives

Sitti, M., Murphy, M., Aksak, B.

March 2013, US Patent App. 13/845,702 (misc)

pi

[BibTex]

[BibTex]


no image
Action and Goal Related Decision Variables Modulate the Competition Between Multiple Potential Targets

Enachescu, V, Christopoulos, Vassilios N, Schrater, P. R., Schaal, S.

In Abstracts of Neural Control of Movement Conference (NCM 2013), February 2013 (inproceedings)

am

[BibTex]

[BibTex]


no image
Optimal control of reaching includes kinematic constraints

Mistry, M., Theodorou, E., Schaal, S., Kawato, M.

Journal of Neurophysiology, 2013, clmc (article)

Abstract
We investigate adaptation under a reaching task with an acceleration-based force field perturbation designed to alter the nominal straight hand trajectory in a potentially benign manner:pushing the hand of course in one direction before subsequently restoring towards the target. In this particular task, an explicit strategy to reduce motor effort requires a distinct deviation from the nominal rectilinear hand trajectory. Rather, our results display a clear directional preference during learning, as subjects adapted perturbed curved trajectories towards their initial baselines. We model this behavior using the framework of stochastic optimal control theory and an objective function that trades-of the discordant requirements of 1) target accuracy, 2) motor effort, and 3) desired trajectory. Our work addresses the underlying objective of a reaching movement, and we suggest that robustness, particularly against internal model uncertainly, is as essential to the reaching task as terminal accuracy and energy effciency.

am

PDF [BibTex]

PDF [BibTex]


Fusing visual and tactile sensing for 3-D object reconstruction while grasping
Fusing visual and tactile sensing for 3-D object reconstruction while grasping

Ilonen, J., Bohg, J., Kyrki, V.

In IEEE International Conference on Robotics and Automation (ICRA), pages: 3547-3554, 2013 (inproceedings)

Abstract
In this work, we propose to reconstruct a complete 3-D model of an unknown object by fusion of visual and tactile information while the object is grasped. Assuming the object is symmetric, a first hypothesis of its complete 3-D shape is generated from a single view. This initial model is used to plan a grasp on the object which is then executed with a robotic manipulator equipped with tactile sensors. Given the detected contacts between the fingers and the object, the full object model including the symmetry parameters can be refined. This refined model will then allow the planning of more complex manipulation tasks. The main contribution of this work is an optimal estimation approach for the fusion of visual and tactile data applying the constraint of object symmetry. The fusion is formulated as a state estimation problem and solved with an iterative extended Kalman filter. The approach is validated experimentally using both artificial and real data from two different robotic platforms.

am

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors

Ijspeert, A., Nakanishi, J., Pastor, P., Hoffmann, H., Schaal, S.

Neural Computation, (25):328-373, 2013, clmc (article)

Abstract
Nonlinear dynamical systems have been used in many disciplines to model complex behaviors, including biological motor control, robotics, perception, economics, traffic prediction, and neuroscience. While often the unexpected emergent behavior of nonlinear systems is the focus of investigations, it is of equal importance to create goal-directed behavior (e.g., stable locomotion from a system of coupled oscillators under perceptual guidance). Modeling goal-directed behavior with nonlinear systems is, however, rather difficult due to the parameter sensitivity of these systems, their complex phase transitions in response to subtle parameter changes, and the difficulty of analyzing and predicting their long-term behavior; intuition and time-consuming parameter tuning play a major role. This letter presents and reviews dynamical movement primitives, a line of research for modeling attractor behaviors of autonomous nonlinear dynamical systems with the help of statistical learning techniques. The essence of our approach is to start with a simple dynamical system, such as a set of linear differential equations, and transform those into a weakly nonlinear system with prescribed attractor dynamics by meansof a learnable autonomous forcing term. Both point attractors and limit cycle attractors of almost arbitrary complexity can be generated. We explain the design principle of our approach and evaluate its properties in several example applications in motor control and robotics.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Learning Objective Functions for Manipulation

Kalakrishnan, M., Pastor, P., Righetti, L., Schaal, S.

In 2013 IEEE International Conference on Robotics and Automation, IEEE, Karlsruhe, Germany, 2013 (inproceedings)

Abstract
We present an approach to learning objective functions for robotic manipulation based on inverse reinforcement learning. Our path integral inverse reinforcement learning algorithm can deal with high-dimensional continuous state-action spaces, and only requires local optimality of demonstrated trajectories. We use L 1 regularization in order to achieve feature selection, and propose an efficient algorithm to minimize the resulting convex objective function. We demonstrate our approach by applying it to two core problems in robotic manipulation. First, we learn a cost function for redundancy resolution in inverse kinematics. Second, we use our method to learn a cost function over trajectories, which is then used in optimization-based motion planning for grasping and manipulation tasks. Experimental results show that our method outperforms previous algorithms in high-dimensional settings.

am mg

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Angular Motion Control Using a Closed-Loop CPG for a Water-Running Robot

Thatte, N., Khoramshahi, M., Ijspeert, A., Sitti, M.

In Dynamic Walking 2013, (EPFL-CONF-199763), 2013 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Two-dimensional magnetic micro-module reconfigurations based on inter-modular interactions

Miyashita, S., Diller, E., Sitti, M.

The International Journal of Robotics Research, 32(5):591-613, SAGE Publications Sage UK: London, England, 2013 (article)

pi

[BibTex]

[BibTex]


no image
Contact compliance effects in the frictional response of bioinspired fibrillar adhesives

Piccardo, M., Chateauminois, A., Fretigny, C., Pugno, N. M., Sitti, M.

Journal of The Royal Society Interface, 10(83):20130182, The Royal Society, 2013 (article)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Co-chairs

VINCENT, Julian, ZHU, Di, DAI, Zhendong, CHEN, Da, JIANG, Lei, KANG, Le, REN, Luquan, XUE, Qunji, Zhao, Chunsheng, BARNES, Jon, others

2013 (article)

pi

[BibTex]

[BibTex]


no image
Light-induced microbubble poration of localized cells

Fan, Qihui, Hu, Wenqi, Ohta, Aaron T

In Engineering in Medicine and Biology Society (EMBC), 2013 35th Annual International Conference of the IEEE, pages: 4482-4485, 2013 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Using Torque Redundancy to Optimize Contact Forces in Legged Robots

Righetti, L., Buchli, J., Mistry, M., Kalakrishnan, M., Schaal, S.

In Redundancy in Robot Manipulators and Multi-Robot Systems, 57, pages: 35-51, Lecture Notes in Electrical Engineering, Springer Berlin Heidelberg, 2013 (incollection)

Abstract
The development of legged robots for complex environments requires controllers that guarantee both high tracking performance and compliance with the environment. More specifically the control of contact interaction with the environment is of crucial importance to ensure stable, robust and safe motions. In the following, we present an inverse dynamics controller that exploits torque redundancy to directly and explicitly minimize any combination of linear and quadratic costs in the contact constraints and in the commands. Such a result is particularly relevant for legged robots as it allows to use torque redundancy to directly optimize contact interactions. For example, given a desired locomotion behavior, it can guarantee the minimization of contact forces to reduce slipping on difficult terrains while ensuring high tracking performance of the desired motion. The proposed controller is very simple and computationally efficient, and most importantly it can greatly improve the performance of legged locomotion on difficult terrains as can be seen in the experimental results.

am mg

link (url) [BibTex]

link (url) [BibTex]


no image
Topological Control of Cell Sheet Migration by the 3D Microenvironment

Song, J., Kim, Y. T., Hazar, M., LeDuc, P. R., Davidson, L. A., Sitti, M.

Biophysical Journal, 104(2):147a, Elsevier, 2013 (article)

pi

[BibTex]

[BibTex]


no image
Independent control of multiple magnetic microrobots in three dimensions

Diller, E., Giltinan, J., Sitti, M.

The International Journal of Robotics Research, 32(5):614-631, SAGE Publications Sage UK: London, England, 2013 (article)

pi

[BibTex]

[BibTex]


no image
A hybrid topological and structural optimization method to design a 3-DOF planar motion compliant mechanism

Lum, G. Z., Teo, T. J., Yang, G., Yeo, S. H., Sitti, M.

In Advanced Intelligent Mechatronics (AIM), 2013 IEEE/ASME International Conference on, pages: 247-254, 2013 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Modular micro-robotic assembly through magnetic actuation and thermal bonding

Diller, E., Zhang, N., Sitti, M.

Journal of Micro-Bio Robotics, 8(3-4):121-131, Springer Berlin Heidelberg, 2013 (article)

pi

[BibTex]

[BibTex]


no image
A simulation and design tool for a passive rotation flapping wing mechanism

Arabagi, V., Hines, L., Sitti, M.

IEEE/ASME Transactions on Mechatronics, 18(2):787-798, 2013 (article)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
GECKO-INSPIRED POLYMER ADHESIVES

Menguc, Yigit, Metin, Metin

Polymer Adhesion, Friction, and Lubrication, pages: 351, Wiley, 2013 (article)

pi

[BibTex]

[BibTex]


no image
Near and far-wall effects on the three-dimensional motion of bacteria-driven microbeads

Edwards, M. R., Wright Carlsen, R., Sitti, M.

Applied Physics Letters, 102(14):143701, AIP, 2013 (article)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
SoftCubes: towards a soft modular matter

Yim, S., Sitti, M.

In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages: 530-536, 2013 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Magnetically Actuated Soft Capsule With the Multimodal Drug Release Function

Yim, S., Goyal, K., Sitti, M.

IEEE/ASME Trans. on Mechatronics, 18(4):1413-1418, IEEE, 2013 (article)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Optimal distribution of contact forces with inverse-dynamics control

Righetti, L., Buchli, J., Mistry, M., Kalakrishnan, M., Schaal, S.

The International Journal of Robotics Research, 32(3):280-298, March 2013 (article)

Abstract
The development of legged robots for complex environments requires controllers that guarantee both high tracking performance and compliance with the environment. More specifically the control of the contact interaction with the environment is of crucial importance to ensure stable, robust and safe motions. In this contribution we develop an inverse-dynamics controller for floating-base robots under contact constraints that can minimize any combination of linear and quadratic costs in the contact constraints and the commands. Our main result is the exact analytical derivation of the controller. Such a result is particularly relevant for legged robots as it allows us to use torque redundancy to directly optimize contact interactions. For example, given a desired locomotion behavior, we can guarantee the minimization of contact forces to reduce slipping on difficult terrains while ensuring high tracking performance of the desired motion. The main advantages of the controller are its simplicity, computational efficiency and robustness to model inaccuracies. We present detailed experimental results on simulated humanoid and quadruped robots as well as a real quadruped robot. The experiments demonstrate that the controller can greatly improve the robustness of locomotion of the robots.1

am mg

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Tank-like module-based climbing robot using passive compliant joints

Seo, T., Sitti, M.

IEEE/ASME Transactions on Mechatronics, 18(1):397-408, 2013 (article)

pi

[BibTex]

[BibTex]


no image
Flapping wings via direct-driving by DC motors

Azhar, M., Campolo, D., Lau, G., Hines, L., Sitti, M.

In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages: 1397-1402, 2013 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Three dimensional independent control of multiple magnetic microrobots

Diller, E., Giltinan, J., Jena, P., Sitti, M.

In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages: 2576-2581, 2013 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Enhanced fabrication and characterization of gecko-inspired mushroom-tipped microfiber adhesives

Song, J., Mengüç, Y., Sitti, M.

Journal of Adhesion Science and Technology, 27(17):1921-1932, Routledge, 2013 (article)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
A Perching Mechanism for Flying Robots Using a Fibre-Based Adhesive

Daler, L., Klaptocz, A., Briod, A., Sitti, M., Floreano, D.

In Robotics and Automation (ICRA), 2013 IEEE International Conference on, 2013 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Micro-scale mobile robotics

Diller, E., Sitti, M.

Foundations and Trends in Robotics, 2(3):143-259, Now Publishers Incorporated, 2013 (article)

pi

[BibTex]

[BibTex]


no image
Survey and Introduction to the Focused Section on Bio-Inspired Mechatronics

Sitti, M., Menciassi, A., Ijspeert, A., Low, K. H., Kim, S.

Mechatronics, IEEE/ASME Transactions on, 18(2):409-418, DOI: 10.1109/TMECH.2012. 2233492, 2013 (article)

pi

[BibTex]

[BibTex]


no image
Bonding methods for modular micro-robotic assemblies

Diller, E., Zhang, N., Sitti, M.

In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages: 2588-2593, 2013 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Learning Task Error Models for Manipulation

Pastor, P., Kalakrishnan, M., Binney, J., Kelly, J., Righetti, L., Sukhatme, G. S., Schaal, S.

In 2013 IEEE Conference on Robotics and Automation, IEEE, Karlsruhe, Germany, 2013 (inproceedings)

Abstract
Precise kinematic forward models are important for robots to successfully perform dexterous grasping and manipulation tasks, especially when visual servoing is rendered infeasible due to occlusions. A lot of research has been conducted to estimate geometric and non-geometric parameters of kinematic chains to minimize reconstruction errors. However, kinematic chains can include non-linearities, e.g. due to cable stretch and motor-side encoders, that result in significantly different errors for different parts of the state space. Previous work either does not consider such non-linearities or proposes to estimate non-geometric parameters of carefully engineered models that are robot specific. We propose a data-driven approach that learns task error models that account for such unmodeled non-linearities. We argue that in the context of grasping and manipulation, it is sufficient to achieve high accuracy in the task relevant state space. We identify this relevant state space using previously executed joint configurations and learn error corrections for those. Therefore, our system is developed to generate subsequent executions that are similar to previous ones. The experiments show that our method successfully captures the non-linearities in the head kinematic chain (due to a counterbalancing spring) and the arm kinematic chains (due to cable stretch) of the considered experimental platform, see Fig. 1. The feasibility of the presented error learning approach has also been evaluated in independent DARPA ARM-S testing contributing to successfully complete 67 out of 72 grasping and manipulation tasks.

am mg

link (url) DOI [BibTex]

link (url) DOI [BibTex]

2011


no image
Learning, planning, and control for quadruped locomotion over challenging terrain

Kalakrishnan, Mrinal, Buchli, Jonas, Pastor, Peter, Mistry, Michael, Schaal, S.

International Journal of Robotics Research, 30(2):236-258, February 2011 (article)

am

[BibTex]

2011


[BibTex]


no image
STOMP: Stochastic trajectory optimization for motion planning

Kalakrishnan, M., Chitta, S., Theodorou, E., Pastor, P., Schaal, S.

In IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9-13, 2011, clmc (inproceedings)

Abstract
We present a new approach to motion planning using a stochastic trajectory optimization framework. The approach relies on generating noisy trajectories to explore the space around an initial (possibly infeasible) trajectory, which are then combined to produced an updated trajectory with lower cost. A cost function based on a combination of obstacle and smoothness cost is optimized in each iteration. No gradient information is required for the particular optimization algorithm that we use and so general costs for which derivatives may not be available (e.g. costs corresponding to constraints and motor torques) can be included in the cost function. We demonstrate the approach both in simulation and on a dual-arm mobile manipulation system for unconstrained and constrained tasks. We experimentally show that the stochastic nature of STOMP allows it to overcome local minima that gradient-based optimizers like CHOMP can get stuck in.

am

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


no image
Development of a Low-Pressure Fluidic Servo-Valve for Wearable Haptic Interfaces and Lightweight Robotic Systems"

Folgheraiter, M., Jordan, M., Benitez, L. M. V., Grimminger, F., Schmidt, S., Albiez, J., Kirchner, F.

In Informatics in Control, Automation and Robotics, pages: 239-252, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011 (inproceedings)

Abstract
This document presents a low-pressure servo-valve specifically designed for haptic interfaces and lightweight robotic applications. The device is able to work with hydraulic and pneumatic fluidic sources, operating within a pressure range of (0{\thinspace}−{\thinspace}50 {\textperiodcentered}105Pa). All sensors and electronics were integrated inside the body of the valve, reducing the need for external circuits. Positioning repeatability as well as the capability to fine modulate the hydraulic flow were measured and verified. Furthermore, the static and dynamic behavior of the valve were evaluated for different working conditions, and a non-linear model identified using a recursive Hammerstein-Wiener parameter adaptation algorithm.

am

DOI [BibTex]

DOI [BibTex]


no image
Path Integral Control and Bounded Rationality

Braun, D. A., Ortega, P. A., Theodorou, E., Schaal, S.

In IEEE Symposium on Adaptive Dynamic Programming And Reinforcement Learning (ADPRL), 2011, clmc (inproceedings)

Abstract
Path integral methods [7], [15],[1] have recently been shown to be applicable to a very general class of optimal control problems. Here we examine the path integral formalism from a decision-theoretic point of view, since an optimal controller can always be regarded as an instance of a perfectly rational decision-maker that chooses its actions so as to maximize its expected utility [8]. The problem with perfect rationality is, however, that finding optimal actions is often very difficult due to prohibitive computational resource costs that are not taken into account. In contrast, a bounded rational decision-maker has only limited resources and therefore needs to strike some compromise between the desired utility and the required resource costs [14]. In particular, we suggest an information-theoretic measure of resource costs that can be derived axiomatically [11]. As a consequence we obtain a variational principle for choice probabilities that trades off maximizing a given utility criterion and avoiding resource costs that arise due to deviating from initially given default choice probabilities. The resulting bounded rational policies are in general probabilistic. We show that the solutions found by the path integral formalism are such bounded rational policies. Furthermore, we show that the same formalism generalizes to discrete control problems, leading to linearly solvable bounded rational control policies in the case of Markov systems. Importantly, Bellman?s optimality principle is not presupposed by this variational principle, but it can be derived as a limit case. This suggests that the information- theoretic formalization of bounded rationality might serve as a general principle in control design that unifies a number of recently reported approximate optimal control methods both in the continuous and discrete domain.

am

PDF [BibTex]

PDF [BibTex]


no image
Skill learning and task outcome prediction for manipulation

Pastor, P., Kalakrishnan, M., Chitta, S., Theodorou, E., Schaal, S.

In IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9-13, 2011, clmc (inproceedings)

Abstract
Learning complex motor skills for real world tasks is a hard problem in robotic manipulation that often requires painstaking manual tuning and design by a human expert. In this work, we present a Reinforcement Learning based approach to acquiring new motor skills from demonstration. Our approach allows the robot to learn fine manipulation skills and significantly improve its success rate and skill level starting from a possibly coarse demonstration. Our approach aims to incorporate task domain knowledge, where appropriate, by working in a space consistent with the constraints of a specific task. In addition, we also present an approach to using sensor feedback to learn a predictive model of the task outcome. This allows our system to learn the proprioceptive sensor feedback needed to monitor subsequent executions of the task online and abort execution in the event of predicted failure. We illustrate our approach using two example tasks executed with the PR2 dual-arm robot: a straight and accurate pool stroke and a box flipping task using two chopsticks as tools.

am

link (url) Project Page Project Page [BibTex]

link (url) Project Page Project Page [BibTex]


no image
An Iterative Path Integral Stochastic Optimal Control Approach for Learning Robotic Tasks

Theodorou, E., Stulp, F., Buchli, J., Schaal, S.

In Proceedings of the 18th World Congress of the International Federation of Automatic Control, 2011, clmc (inproceedings)

Abstract
Recent work on path integral stochastic optimal control theory Theodorou et al. (2010a); Theodorou (2011) has shown promising results in planning and control of nonlinear systems in high dimensional state spaces. The path integral control framework relies on the transformation of the nonlinear Hamilton Jacobi Bellman (HJB) partial differential equation (PDE) into a linear PDE and the approximation of its solution via the use of the Feynman Kac lemma. In this work, we are reviewing the generalized version of path integral stochastic optimal control formalism Theodorou et al. (2010a), used for optimal control and planing of stochastic dynamical systems with state dependent control and diffusion matrices. Moreover we present the iterative path integral control approach, the so called Policy Improvement with Path Integrals or (PI2 ) which is capable of scaling in high dimensional robotic control problems. Furthermore we present a convergence analysis of the proposed algorithm and we apply the proposed framework to a variety of robotic tasks. Finally with the goal to perform locomotion the iterative path integral control is applied for learning nonlinear limit cycle attractors with adjustable land scape.

am

PDF [BibTex]

PDF [BibTex]


no image
Design and application of a wire-driven bidirectional telescopic mechanism for workspace expansion with a focus on shipbuilding tasks

Lee, D., Chang, D., Shin, Y., Son, D., Kim, T., Lee, K., Kim, J.

Advanced Robotics, 25, 2011 (article)

pi

[BibTex]

[BibTex]


no image
Bayesian robot system identification with input and output noise

Ting, J., D’Souza, A., Schaal, S.

Neural Networks, 24(1):99-108, 2011, clmc (article)

Abstract
For complex robots such as humanoids, model-based control is highly beneficial for accurate tracking while keeping negative feedback gains low for compliance. However, in such multi degree-of-freedom lightweight systems, conventional identification of rigid body dynamics models using CAD data and actuator models is inaccurate due to unknown nonlinear robot dynamic effects. An alternative method is data-driven parameter estimation, but significant noise in measured and inferred variables affects it adversely. Moreover, standard estimation procedures may give physically inconsistent results due to unmodeled nonlinearities or insufficiently rich data. This paper addresses these problems, proposing a Bayesian system identification technique for linear or piecewise linear systems. Inspired by Factor Analysis regression, we develop a computationally efficient variational Bayesian regression algorithm that is robust to ill-conditioned data, automatically detects relevant features, and identifies input and output noise. We evaluate our approach on rigid body parameter estimation for various robotic systems, achieving an error of up to three times lower than other state-of-the-art machine learning methods

am

link (url) [BibTex]

link (url) [BibTex]


no image
Learning variable impedance control

Buchli, J., Stulp, F., Theodorou, E., Schaal, S.

International Journal of Robotics Research, 2011, clmc (article)

Abstract
One of the hallmarks of the performance, versatility, and robustness of biological motor control is the ability to adapt the impedance of the overall biomechanical system to different task requirements and stochastic disturbances. A transfer of this principle to robotics is desirable, for instance to enable robots to work robustly and safely in everyday human environments. It is, however, not trivial to derive variable impedance controllers for practical high degree-of-freedom (DOF) robotic tasks. In this contribution, we accomplish such variable impedance control with the reinforcement learning (RL) algorithm PISq ({f P}olicy {f I}mprovement with {f P}ath {f I}ntegrals). PISq is a model-free, sampling based learning method derived from first principles of stochastic optimal control. The PISq algorithm requires no tuning of algorithmic parameters besides the exploration noise. The designer can thus fully focus on cost function design to specify the task. From the viewpoint of robotics, a particular useful property of PISq is that it can scale to problems of many DOFs, so that reinforcement learning on real robotic systems becomes feasible. We sketch the PISq algorithm and its theoretical properties, and how it is applied to gain scheduling for variable impedance control. We evaluate our approach by presenting results on several simulated and real robots. We consider tasks involving accurate tracking through via-points, and manipulation tasks requiring physical contact with the environment. In these tasks, the optimal strategy requires both tuning of a reference trajectory emph{and} the impedance of the end-effector. The results show that we can use path integral based reinforcement learning not only for planning but also to derive variable gain feedback controllers in realistic scenarios. Thus, the power of variable impedance control is made available to a wide variety of robotic systems and practical applications.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Iterative path integral stochastic optimal control: Theory and applications to motor control

Theodorou, E. A.

University of Southern California, University of Southern California, Los Angeles, CA, 2011 (phdthesis)

am

PDF [BibTex]

PDF [BibTex]


no image
Learning of grasp selection based on shape-templates

Herzog, A.

Karlsruhe Institute of Technology, 2011 (mastersthesis)

am

[BibTex]

[BibTex]