Hierarchical reinforcement learning for adaptive and autonomous decision-making in robotics

Abstract

In recent years, Reinforcement Learning has been able to solve extremely complex games in simulation, but with limited success in deployment to real-world scenarios. The goal of this work is create an ecosystem in which Reinforcement Learning algorithms can be deployed onto real robots in complex games. The ecosystem begins with the creation of a development pipeline which can be used to progressively train Reinforcement Learning Algorithms in increasingly realistic scenarios, culminating with the deployment of these algorithm onto a real system. The pipeline is paired with the novel Reinforcement Learning algorithms that are better able to adapt to new scenarios than traditional methods for autonomy and robotic planning.We implement two techniques to enable this adaptation. First, we implement a hierarchical Reinforcement Learning architecture that uses differentiated sub-policies governed by a hierarchical controller to enable fast adaptation. Second we introduce a confidence-based training process for the hierarchical controller which improves training stability and convergence times. These algorithmic contributions were evaluated using our development pipeline