Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Sense representations for Portuguese: experiments with sense embeddings
  and deep neural language models   Sense representations have gone beyond word representations like Word2Vec,
GloVe and FastText and achieved innovative performance on a wide range of
natural language processing tasks. Although very useful in many applications,
the traditional approaches for generating word embeddings have a strict
drawback: they produce a single vector representation for a given word ignoring
the fact that ambiguous words can assume different meanings. In this paper, we
explore unsupervised sense representations which, different from traditional
word embeddings, are able to induce different senses of a word by analyzing its
contextual semantics in a text. The unsupervised sense representations
investigated in this paper are: sense embeddings and deep neural language
models. We present the first experiments carried out for generating sense
embeddings for Portuguese. Our experiments show that the sense embedding model
(Sense2vec) outperformed traditional word embeddings in syntactic and semantic
analogies task, proving that the language resource generated here can improve
the performance of NLP tasks in Portuguese. We also evaluated the performance
of pre-trained deep neural language models (ELMo and BERT) in two transfer
learning approaches: feature based and fine-tuning, in the semantic textual
similarity task. Our experiments indicate that the fine tuned Multilingual and
Portuguese BERT language models were able to achieve better accuracy than the
ELMo model and baselines.

26
The categories of this paper are: bayes-an
28
The categories of this paper are: alg-geom
21
The categories of this paper are: adap-org
29
The categories of this paper are: funct-an
23
The categories of this paper are: solv-int
22
The categories of this paper are: chao-dyn
4
The categories of this paper are: cond-mat
19
The categories of this paper are: eess
33
The categories of this paper are: plasm-ph
16
The categories of this paper are: stat
6
The categories of this paper are: astro-ph
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
11
The categories of this paper are: hep-lat
36
The categories of this paper are: cmp-lg
0
The categories of this paper are: hep-ph