Sensible activation for RNNs
Sensible activation for RNNs

Have noticed how default GRUs works usually worse than LSTMs? It seems that "tanh" is a more sensible activation choice. Also for GRUs, tanh seems to be the default:
see http://arxiv.org/pdf/1412.3555v1.pdf Section 3.2
kemaswill/keras,dolaameng/keras,DeepGnosis/keras,kuza55/keras,daviddiazvico/keras,keras-team/keras,relh/keras,nebw/keras,keras-team/keras