Header logo is

Curiosity-driven learning with Context Tree Weighting

2014

Conference Paper

ei


In the first simulation, the intrinsic motivation of the agent was given by measuring learning progress through reduction in informational surprise (Figure 1 A-C). This way the agent should first learn the action that is easiest to learn (a1), and then switch to other actions that still allow for learning (a2) and ignore actions that cannot be learned at all (a3). This is exactly what we found in our simple environment. Compared to the original developmental learning algorithm based on learning progress proposed by Oudeyer [2], our Context Tree Weighting approach does not require local experts to do prediction, rather it learns the conditional probability distribution over observations given action in one structure. In the second simulation, the intrinsic motivation of the agent was given by measuring compression progress through improvement in compressibility (Figure 1 D-F). The agent behaves similarly: the agent first concentrates on the action with the most predictable consequence and then switches over to the regular action where the consequence is more difficult to predict, but still learnable. Unlike the previous simulation, random actions are also interesting to some extent because the compressed symbol strings use 8-bit representations, while only 2 bits are required for our observation space. Our preliminary results suggest that Context Tree Weighting might provide a useful representation to study problems of development.

Author(s): Peng, Z and Braun, DA
Pages: 366-367
Year: 2014
Month: October
Publisher: IEEE

Department(s): Empirical Inference
Bibtex Type: Conference Paper (conference)

DOI: 10.1109/DEVLRN.2014.6983008
Event Name: 4th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (IEEE ICDL-EPIROB 2014)
Event Place: Genova, Italy

Address: Piscataway, NJ, USA

BibTex

@conference{PengB2014,
  title = {Curiosity-driven learning with Context Tree Weighting},
  author = {Peng, Z and Braun, DA},
  pages = {366-367},
  publisher = {IEEE},
  address = {Piscataway, NJ, USA},
  month = oct,
  year = {2014},
  doi = {10.1109/DEVLRN.2014.6983008},
  month_numeric = {10}
}