Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Model Selection for Offline Reinforcement Learning: Practical
  Considerations for Healthcare Settings   Reinforcement learning (RL) can be used to learn treatment policies and aid
decision making in healthcare. However, given the need for generalization over
complex state/action spaces, the incorporation of function approximators (e.g.,
deep neural networks) requires model selection to reduce overfitting and
improve policy performance at deployment. Yet a standard validation pipeline
for model selection requires running a learned policy in the actual
environment, which is often infeasible in a healthcare setting. In this work,
we investigate a model selection pipeline for offline RL that relies on
off-policy evaluation (OPE) as a proxy for validation performance. We present
an in-depth analysis of popular OPE methods, highlighting the additional
hyperparameters and computational requirements (fitting/inference of auxiliary
models) when used to rank a set of candidate policies. We compare the utility
of different OPE methods as part of the model selection pipeline in the context
of learning to treat patients with sepsis. Among all the OPE methods we
considered, fitted Q evaluation (FQE) consistently leads to the best validation
ranking, but at a high computational cost. To balance this trade-off between
accuracy of ranking and computational efficiency, we propose a simple two-stage
approach to accelerate model selection by avoiding potentially unnecessary
computation. Our work serves as a practical guide for offline RL model
selection and can help RL practitioners select policies using real-world
datasets. To facilitate reproducibility and future extensions, the code
accompanying this paper is available online at
https://github.com/MLD3/OfflineRL_ModelSelection.

19
The categories of this paper are: eess
5
The categories of this paper are: gr-qc
21
The categories of this paper are: adap-org
9
The categories of this paper are: nlin
14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
2
The categories of this paper are: math
25
The categories of this paper are: patt-sol
22
The categories of this paper are: chao-dyn
15
The categories of this paper are: nucl-ex
13
The categories of this paper are: quant-ph
36
The categories of this paper are: cmp-lg
4
The categories of this paper are: cond-mat
27
The categories of this paper are: comp-gas
17
The categories of this paper are: q-fin
31
The categories of this paper are: ao-sci