Numerically more stable softmax for attention. Thanks Joost!
Numerically more stable softmax for attention. Thanks Joost!

Seems to fix NaN issues we encountered when models converge, especially with Adam.
shuoyangd/nematus,Proyag/nematus,yang1fan2/nematus,yang1fan2/nematus,yang1fan2/nematus,rsennrich/nematus,EdinburghNLP/nematus,yang1fan2/nematus,shuoyangd/nematus,Proyag/nematus,yang1fan2/nematus,Proyag/nematus,cshanbo/nematus,rsennrich/nematus,EdinburghNLP/nematus,shuoyangd/nematus,yang1fan2/nematus,Proyag/nematus,shuoyangd/nematus,cshanbo/nematus,rsennrich/nematus,EdinburghNLP/nematus,shuoyangd/nematus,cshanbo/nematus,Proyag/nematus,rsennrich/nematus,EdinburghNLP/nematus,Proyag/nematus,yang1fan2/nematus,EdinburghNLP/nematus,yang1fan2/nematus,EdinburghNLP/nematus,rsennrich/nematus,yang1fan2/nematus,cshanbo/nematus,rsennrich/nematus,cshanbo/nematus,yang1fan2/nematus,cshanbo/nematus,shuoyangd/nematus