Deep Surrogate Assisted Generation of Environments
Varun Bhatt*
University of Southern California
vsbhatt@usc.edu
Bryon Tjanaka*
University of Southern California
tjanaka@usc.edu
Matthew C. Fontaine*
University of Southern California
mfontain@usc.edu
Stefanos Nikolaidis
University of Southern California
nikolaid@usc.edu
Abstract
Recent progress in reinforcement learning (RL) has started producing generally capable agents that can solve a distribution of complex environments. These agents are typically tested on fixed, human-authored environments. On the other hand, quality diversity (QD) optimization has been proven to be an effective component of environment generation algorithms, which can generate collections of high-quality environments that are diverse in the resulting agent behaviors. However, these algorithms require potentially expensive simulations of agents on newly generated environments. We propose Deep Surrogate Assisted Generation of Environments (DSAGE), a sample-efficient QD environment generation algorithm that maintains a deep surrogate model for predicting agent behaviors in new environments. Results in two benchmark domains show that DSAGE significantly outperforms existing QD environment generation algorithms in discovering collections of environments that elicit diverse behaviors of a state-of-the-art RL agent and a planning agent.
This website displays videos associated with the figures in the paper. We have arranged the videos to match the figure layouts in the paper as closely as possible. Click on the videos to view them individually.
Figure 1
