An empirical analysis of deep network loss surfaces.

janelia7_blocks-janelia7_biblio_header | block
arXiv. 2016 Dec 13;:arXiv:1612.04010
An empirical analysis of deep network loss surfaces. Branson Lab
Im DJ, Tao M, Branson K
janelia7_blocks-janelia7_biblio_abstract | block

Abstract

The training of deep neural networks is a high-dimension optimization problem with respect to the loss function of a model. Unfortunately, these functions are of high dimension and non-convex and hence difficult to characterize. In this paper, we empirically investigate the geometry of the loss functions for state-of-the-art networks with multiple stochastic optimization methods. We do this through several experiments that are visualized on polygons to understand how and when these stochastic optimization methods find minima.

node:body | entity_field
janelia7_blocks-janelia7_biblio_authors | block

Janelia Authors

Kristin Branson
janelia7_blocks-janelia7_biblio_tools | block