We introduce a new approach to estimate continuous actions using actor-critic algorithms for reinforcement learning problems. Policy gradient methods usually predict one continuous action estimate or parameters of a presumed distribution (most commonly Gaussian) for any given state which might not be optimal as it may not capture the complete description of the target distribution. Our approach instead predicts M actions with the policy network (actor) and then uniformly sample one action during training as well as testing at each state. This allows the agent to learn a simple stochastic policy that has an easy to compute expected return. In all experiments, this facilitates better exploration of the state space during training and converges to a better policy.
We introduce a novel reinforcement learning algorithm, that predicts multiple actions and samples from them. This work introduces a uniform mixture of deterministic policies, and find that this parametrization of stochastic policies outperforms DDPG on several OpenAI gym benchmarks. The authors investigate a method for improving the performance of networks trained with DDPG, and show improved performance on a large number of standard continuous control environment.