Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Causal Transformer for Estimating Counterfactual Outcomes   Estimating counterfactual outcomes over time from observational data is
relevant for many applications (e.g., personalized medicine). Yet,
state-of-the-art methods build upon simple long short-term memory (LSTM)
networks, thus rendering inferences for complex, long-range dependencies
challenging. In this paper, we develop a novel Causal Transformer for
estimating counterfactual outcomes over time. Our model is specifically
designed to capture complex, long-range dependencies among time-varying
confounders. For this, we combine three transformer subnetworks with separate
inputs for time-varying covariates, previous treatments, and previous outcomes
into a joint network with in-between cross-attentions. We further develop a
custom, end-to-end training procedure for our Causal Transformer. Specifically,
we propose a novel counterfactual domain confusion loss to address confounding
bias: it aims to learn adversarial balanced representations, so that they are
predictive of the next outcome but non-predictive of the current treatment
assignment. We evaluate our Causal Transformer based on synthetic and
real-world datasets, where it achieves superior performance over current
baselines. To the best of our knowledge, this is the first work proposing
transformer-based architecture for estimating counterfactual outcomes from
longitudinal data.

0
The categories of this paper are: hep-ph
21
The categories of this paper are: adap-org
25
The categories of this paper are: patt-sol
2
The categories of this paper are: math
9
The categories of this paper are: nlin
8
The categories of this paper are: hep-ex
19
The categories of this paper are: eess
5
The categories of this paper are: gr-qc
28
The categories of this paper are: alg-geom
14
The categories of this paper are: math-ph
18
The categories of this paper are: econ
36
The categories of this paper are: cmp-lg
34
The categories of this paper are: chem-ph
33
The categories of this paper are: plasm-ph
23
The categories of this paper are: solv-int
3
The categories of this paper are: physics