refactored some things from stupidly translated list structures to more sensible stack structures because backpropagation is stack-based; renamed "nabla" to "del" because my mental model always calls it "del"; eliminated the first derivative of sigmoid by memoizing the results of using the activations (as the sigmoid' of some input z is the same as the corresponding activation a * (1 - a)) These optimizations have netted a small performance gain but profiling has always revealed that basically all of our time is spent doing matrix math so we didn't really gain much
refactored some things from stupidly translated list structures to more sensible stack structures because backpropagation is stack-based; renamed "nabla" to "del" because my mental model always calls it "del"; eliminated the first derivative of sigmoid by memoizing the results of using the activations (as the sigmoid' of some input z is the same as the corresponding activation a * (1 - a))
These optimizations have netted a small performance gain but profiling has always revealed that basically all of our time is spent doing matrix math so we didn't really gain much

akindle/garbage-nn