Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Reinforcement Learning on Web Interfaces Using Workflow-Guided
  Exploration   Reinforcement learning (RL) agents improve through trial-and-error, but when
reward is sparse and the agent cannot discover successful action sequences,
learning stagnates. This has been a notable problem in training deep RL agents
to perform web-based tasks, such as booking flights or replying to emails,
where a single mistake can ruin the entire sequence of actions. A common remedy
is to "warm-start" the agent by pre-training it to mimic expert demonstrations,
but this is prone to overfitting. Instead, we propose to constrain exploration
using demonstrations. From each demonstration, we induce high-level "workflows"
which constrain the allowable actions at each time step to be similar to those
in the demonstration (e.g., "Step 1: click on a textbox; Step 2: enter some
text"). Our exploration policy then learns to identify successful workflows and
samples actions that satisfy these workflows. Workflows prune out bad
exploration directions and accelerate the agent's ability to discover rewards.
We use our approach to train a novel neural policy designed to handle the
semi-structured nature of websites, and evaluate on a suite of web tasks,
including the recent World of Bits benchmark. We achieve new state-of-the-art
results, and show that workflow-guided exploration improves sample efficiency
over behavioral cloning by more than 100x.

33
The categories of this paper are: plasm-ph
14
The categories of this paper are: math-ph
9
The categories of this paper are: nlin
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys
17
The categories of this paper are: q-fin
28
The categories of this paper are: alg-geom
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
10
The categories of this paper are: q-bio
19
The categories of this paper are: eess
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
18
The categories of this paper are: econ
15
The categories of this paper are: nucl-ex