Header logo is

Reinforcement Learning for Parameterized Motor Primitives


Conference Paper



One of the major challenges in both action generation for robotics and in the understanding of human motor control is to learn the "building blocks of movement generation", called motor primitives. Motor primitives, as used in this paper, are parameterized control policies such as splines or nonlinear differential equations with desired attractor properties. While a lot of progress has been made in teaching parameterized motor primitives using supervised or imitation learning, the self-improvement by interaction of the system with the environment remains a challenging problem. In this paper, we evaluate different reinforcement learning approaches for improving the performance of parameterized motor primitives. For pursuing this goal, we highlight the difficulties with current reinforcement learning methods, and outline both established and novel algorithms for the gradient-based improvement of parameterized policies. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm.

Author(s): Peters, J. and Schaal, S.
Book Title: Proceedings of the 2006 International Joint Conference on Neural Networks
Pages: 73-80
Year: 2006

Department(s): Autonomous Motion, Empirical Inference
Bibtex Type: Conference Paper (inproceedings)

DOI: 10.1109/IJCNN.2006.246662
Event Name: IJCNN 2006
Event Place: Los Angeles, CA, USA

Cross Ref: p2656
Note: clmc
URL: http://www-clmc.usc.edu/publications/P/peters-IJCNN2006.pdf


  title = {Reinforcement Learning for Parameterized Motor Primitives},
  author = {Peters, J. and Schaal, S.},
  booktitle = {Proceedings of the 2006 International Joint Conference on Neural Networks},
  pages = {73-80},
  year = {2006},
  note = {clmc},
  crossref = {p2656},
  url = {http://www-clmc.usc.edu/publications/P/peters-IJCNN2006.pdf}