Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Active Deep Q-learning with Demonstration   Recent research has shown that although Reinforcement Learning (RL) can
benefit from expert demonstration, it usually takes considerable efforts to
obtain enough demonstration. The efforts prevent training decent RL agents with
expert demonstration in practice. In this work, we propose Active Reinforcement
Learning with Demonstration (ARLD), a new framework to streamline RL in terms
of demonstration efforts by allowing the RL agent to query for demonstration
actively during training. Under the framework, we propose Active Deep
Q-Network, a novel query strategy which adapts to the dynamically-changing
distributions during the RL training process by estimating the uncertainty of
recent states. The expert demonstration data within Active DQN are then
utilized by optimizing supervised max-margin loss in addition to temporal
difference loss within usual DQN training. We propose two methods of estimating
the uncertainty based on two state-of-the-art DQN models, namely the divergence
of bootstrapped DQN and the variance of noisy DQN. The empirical results
validate that both methods not only learn faster than other passive expert
demonstration methods with the same amount of demonstration and but also reach
super-expert level of performance across four different tasks.

20
The categories of this paper are: acc-phys
15
The categories of this paper are: nucl-ex
12
The categories of this paper are: nucl-th
8
The categories of this paper are: hep-ex
35
The categories of this paper are: mtrl-th
7
The categories of this paper are: hep-th
22
The categories of this paper are: chao-dyn
37
The categories of this paper are: supr-con
2
The categories of this paper are: math
36
The categories of this paper are: cmp-lg
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
18
The categories of this paper are: econ
25
The categories of this paper are: patt-sol
26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org