An empirical analysis of deep network loss surfaces.

janelia7_blocks-janelia7_biblio_header | block

arXiv. 2016 Dec 13;:arXiv:1612.04010

An empirical analysis of deep network loss surfaces. Branson Lab

Im DJ, Tao M, Branson K

janelia7_blocks-janelia7_biblio_abstract | block

Abstract

The training of deep neural networks is a high-dimension optimization problem with respect to the loss function of a model. Unfortunately, these functions are of high dimension and non-convex and hence difficult to characterize. In this paper, we empirically investigate the geometry of the loss functions for state-of-the-art networks with multiple stochastic optimization methods. We do this through several experiments that are visualized on polygons to understand how and when these stochastic optimization methods find minima.

node:body | entity_field

Main Menu (Mobile)- Block

Main Menu - Block

An empirical analysis of deep network loss surfaces.

Abstract

Janelia Authors