Distributional Properties of ReLU-Activations in Artificial Neural Networks that Learn by Memorization

Abstract

We investigate the distributional properties of layers in Artificial Neural Network (ANN) that can be used to distinguish between networks learning by generalization and memorizing networks. Starting from the notion of memorization being essentially definable as learning ‘rare’ features of the input data, we propose the activation probability of Rectified Linear Units (ReLU)-neurons as an important indicator of memorization. Building on this idea, we show how hypotheses about distributional properties over whole networks structures can be derived from the activation probability of singular neurons in memorizing ANNs. We such extend previous work on using Poisson process models of activations in ANN by considering the correlation between neurons. Using this approach, we further simulate the effect of memorizing neurons on distributional properties of weight matrices and activation magnitudes and find a connection between L1/L2-norm regularization of weight matrices. We empirically evaluate the distributions of activation rate, correlation structure and weight matrices in memorizing and generalizing ANNs on a simple MNIST-classification task. Our initial findings show how the activation frequency and intra-layer correlation structure can be used to distinguish generalizing from memorizing networks and for inferring distributional properties on affected parts of the networks. This work presents a building block to later derive online metrics for memorization in ANNs.