learning rate is too high and saturates all those hard sigmoids and then learning dies. using lr = 0.0001 produces no NAN so far
learning rate is too high and saturates all those hard sigmoids and then learning dies. using lr = 0.0001 produces no NAN so far

# sgd = SGD(lr=0.0001, decay=1e-6, momentum=0.9, nesterov=True, clipgrad=0.5)
lukovkin/ufcnn-keras,lukovkin/ufcnn-keras