Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A pseudo-parallel Python environment for database curation   One of the major challenges providing large databases like the WFCAM Science
Archive (WSA) is to minimize ingest times for pixel/image metadata and
catalogue data. In this article we describe how the pipeline processed data are
ingested into the database as the first stage in building a release database
which will be succeeded by advanced processing (source merging, seaming,
detection quality flagging etc.). To accomplish the ingestion procedure as fast
as possible we use a mixed Python/C++ environment and run the required tasks in
a simple parallel modus operandi where the data are split into daily chunks and
then processed on different computers. The created data files can be ingested
into the database immediately as they are available. This flexible way of
handling the data allows the most usage of the available CPUs as the comparison
with sequential processing shows.

7
The categories of this paper are: hep-th
25
The categories of this paper are: patt-sol
12
The categories of this paper are: nucl-th
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
27
The categories of this paper are: comp-gas
30
The categories of this paper are: q-alg
29
The categories of this paper are: funct-an
14
The categories of this paper are: math-ph
34
The categories of this paper are: chem-ph
21
The categories of this paper are: adap-org
18
The categories of this paper are: econ
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
22
The categories of this paper are: chao-dyn
11
The categories of this paper are: hep-lat