Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Comparing Offline and Online Testing of Deep Neural Networks: An
  Autonomous Car Case Study   There is a growing body of research on developing testing techniques for Deep
Neural Networks (DNN). We distinguish two general modes of testing for DNNs:
Offline testing where DNNs are tested as individual units based on test
datasets obtained independently from the DNNs under test, and online testing
where DNNs are embedded into a specific application and tested in a close-loop
mode in interaction with the application environment. In addition, we identify
two sources for generating test datasets for DNNs: Datasets obtained from
real-life and datasets generated by simulators. While offline testing can be
used with datasets obtained from either sources, online testing is largely
confined to using simulators since online testing within real-life applications
can be time-consuming, expensive and dangerous. In this paper, we study the
following two important questions aiming to compare test datasets and testing
modes for DNNs: First, can we use simulator-generated data as a reliable
substitute to real-world data for the purpose of DNN testing? Second, how do
online and offline testing results differ and complement each other? Though
these questions are generally relevant to all autonomous systems, we study them
in the context of automated driving systems where, as study subjects, we use
DNNs automating end-to-end control of cars' steering actuators. Our results
show that simulator-generated datasets are able to yield DNN prediction errors
that are similar to those obtained by testing DNNs with real-life datasets.
Further, offline testing is more optimistic than online testing as many safety
violations identified by online testing could not be identified by offline
testing, while large prediction errors generated by offline testing always led
to severe safety violations detectable by online testing.

27
The categories of this paper are: comp-gas
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
21
The categories of this paper are: adap-org
26
The categories of this paper are: bayes-an
10
The categories of this paper are: q-bio
24
The categories of this paper are: dg-ga
18
The categories of this paper are: econ
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th