DMCA
Intriguing properties of neural networks (2013)
| Citations: | 21 - 1 self |
BibTeX
@MISC{Szegedy13intriguingproperties,
author = {Christian Szegedy and Google Inc and Wojciech Zaremba and Ilya Sutskever and Google Inc and Joan Bruna and Dumitru Erhan and Google Inc and Ian Goodfellow and Rob Fergus},
title = {Intriguing properties of neural networks},
year = {2013}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Deep neural networks are highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. While their expressiveness is the reason they succeed, it also causes them to learn uninter-pretable solutions that could have counter-intuitive properties. In this paper we report two such properties. First, we find that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. It suggests that it is the space, rather than the individual units, that contains the semantic information in the high layers of neural networks. Second, we find that deep neural networks learn input-output mappings that are fairly discontinuous to a significant extent. We can cause the network to misclas-sify an image by applying a certain hardly perceptible perturbation, which is found by maximizing the network’s prediction error. In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input. 1
Keyphrases
neural network deep neural network individual unit different subset various method unit analysis input-output mapping counter-intuitive property visual recognition task high level unit different network random linear combination expressive model network prediction error significant extent uninter-pretable solution individual high level unit perceptible perturbation high layer random artifact specific nature semantic information art performance
