The vast majority of natural sensory data is temporally redundant. For instance, video frames or audio samples which are sampled at nearby points in time tend to have similar values.   Typically, deep learning algorithms take no advantage of this redundancy to reduce computations.   This can be an obscene waste of energy.   We present a variant on backpropagation for neural networks in which computation scales with the rate of change of the data - not the rate at which we process the data.   We do this by implementing a form of Predictive Coding wherein neurons communicate a combination of their state, and their temporal change in state, and quantize this signal using Sigma-Delta modulation.   Intriguingly, this simple communication rule give rise to units that resemble biologically-inspired leaky integrate-and-fire neurons, and to a spike-timing-dependent weight-update similar to Spike-Timing Dependent Plasticity (STDP), a synaptic learning rule observed in the brain.   We demonstrate that on MNIST, on a temporal variant of MNIST, and on Youtube-BB, a dataset with videos in the wild, our algorithm performs about as well as a standard deep network trained with backpropagation, despite only communicating discrete values between layers.  
An algorithm for training neural networks efficiently on temporally redundant data. The paper describes a neural coding scheme for spike based learning in deep neural networks 
This paper presents a method for spike based learning that aims at reducing the needed computation during learning and testing when classifying temporal redundant data. This paper applies a predictive coding version of the Sigma-Delta encoding scheme to reduce a computational load on a deep learning network, combining the three components in a way not seen previously.