Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Visualized Insights into the Optimization Landscape of Fully
  Convolutional Networks   Many image processing tasks involve image-to-image mapping, which can be
addressed well by fully convolutional networks (FCN) without any heavy
preprocessing. Although empirically designing and training FCNs can achieve
satisfactory results, reasons for the improvement in performance are slightly
ambiguous. Our study is to make progress in understanding their generalization
abilities through visualizing the optimization landscapes. The visualization of
objective functions is obtained by choosing a solution and projecting its
vicinity onto a 3D space. We compare three FCN-based networks (two existing
models and a new proposed in this paper for comparison) on multiple datasets.
It has been observed in practice that the connections from the pre-pooled
feature maps to the post-upsampled can achieve better results. We investigate
the cause and provide experiments to shows that the skip-layer connections in
FCN can promote flat optimization landscape, which is well known to generalize
better. Additionally, we explore the relationship between the models
generalization ability and loss surface under different batch sizes. Results
show that large-batch training makes the model converge to sharp minimizers
with chaotic vicinities while small-batch method leads the model to flat
minimizers with smooth and nearly convex regions. Our work may contribute to
insights and analysis for designing and training FCNs.

13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
37
The categories of this paper are: supr-con
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph
19
The categories of this paper are: eess
17
The categories of this paper are: q-fin
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
11
The categories of this paper are: hep-lat
5
The categories of this paper are: gr-qc