Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Pre-trained Language Model based Ranking in Baidu Search   As the heart of a search engine, the ranking system plays a crucial role in
satisfying users' information demands. More recently, neural rankers fine-tuned
from pre-trained language models (PLMs) establish state-of-the-art ranking
effectiveness. However, it is nontrivial to directly apply these PLM-based
rankers to the large-scale web search system due to the following challenging
issues:(1) the prohibitively expensive computations of massive neural PLMs,
especially for long texts in the web-document, prohibit their deployments in an
online ranking system that demands extremely low latency;(2) the discrepancy
between existing ranking-agnostic pre-training objectives and the ad-hoc
retrieval scenarios that demand comprehensive relevance modeling is another
main barrier for improving the online ranking system;(3) a real-world search
engine typically involves a committee of ranking components, and thus the
compatibility of the individually fine-tuned ranking model is critical for a
cooperative ranking system. In this work, we contribute a series of
successfully applied techniques in tackling these exposed issues when deploying
the state-of-the-art Chinese pre-trained language model, i.e., ERNIE, in the
online search engine system. We first articulate a novel practice to
cost-efficiently summarize the web document and contextualize the resultant
summary content with the query using a cheap yet powerful Pyramid-ERNIE
architecture. Then we endow an innovative paradigm to finely exploit the
large-scale noisy and biased post-click behavioral data for relevance-oriented
pre-training. We also propose a human-anchored fine-tuning strategy tailored
for the online ranking system, aiming to stabilize the ranking signals across
various online components. Extensive offline and online experimental results
show that the proposed techniques significantly boost the search engine's
performance.

10
The categories of this paper are: q-bio
2
The categories of this paper are: math
27
The categories of this paper are: comp-gas
30
The categories of this paper are: q-alg
9
The categories of this paper are: nlin
11
The categories of this paper are: hep-lat
0
The categories of this paper are: hep-ph
36
The categories of this paper are: cmp-lg
32
The categories of this paper are: atom-ph
31
The categories of this paper are: ao-sci
17
The categories of this paper are: q-fin
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
37
The categories of this paper are: supr-con
22
The categories of this paper are: chao-dyn
14
The categories of this paper are: math-ph