Policy Reuse (PR) provides Reinforcement Learning algorithms with a mechanism to bias an exploration process by reusing a set of past policies. Policy Reuse offers the challenge of balancing the exploitation of the ongoing learned policy, the exploration of new random actions, and the exploitation of past policies. Efficient application of Policy Reuse requires a mechanism to build, for each domain, a library of policies which is useful and accurate enough to efficiently solve any task in such domain. In this work, we propose a mechanism to create a library of policies based on a similarity metric among policies. If the new policy is similar to any of the past ones, it is not added to the library. Otherwise, it is stored together with the other policies, so it can be reused in the future. Thus, the Policy Library stores the basis or eigen-policies of each domain, i.e., the core past policies that are effectively reusable. Empirical results demonstrate that the Policy Library can be efficiently created and that the stored eigen-policies can be understood as a representation of the structure of the domain.

14 pages

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by reports@cs.cmu.edu