Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning a Word-Level Language Model with Sentence-Level Noise
  Contrastive Estimation for Contextual Sentence Probability Estimation   Inferring the probability distribution of sentences or word sequences is a
key process in natural language processing. While word-level language models
(LMs) have been widely adopted for computing the joint probabilities of word
sequences, they have difficulty in capturing a context long enough for sentence
probability estimation (SPE). To overcome this, recent studies introduced
training methods using sentence-level noise-contrastive estimation (NCE) with
recurrent neural networks (RNNs). In this work, we attempt to extend it for
contextual SPE, which aims to estimate a conditional sentence probability given
a previous text. The proposed NCE samples negative sentences independently of a
previous text so that the trained model gives higher probabilities to the
sentences that are more consistent with \textcolor{blue}{the} context. We apply
our method to a simple word-level RNN LM to focus on the effect of the
sentence-level NCE training rather than on the network architecture. The
quality of estimation was evaluated against multiple-choice cloze-style
questions including both human and automatically generated questions. The
experimental results show that the proposed method improved the SPE quality for
the word-level RNN LM.

18
The categories of this paper are: econ
21
The categories of this paper are: adap-org
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
36
The categories of this paper are: cmp-lg
5
The categories of this paper are: gr-qc
24
The categories of this paper are: dg-ga
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
16
The categories of this paper are: stat
19
The categories of this paper are: eess
11
The categories of this paper are: hep-lat
0
The categories of this paper are: hep-ph
34
The categories of this paper are: chem-ph