New "Adafactor" optimizer - more memory efficient than Adam.
New "Adafactor" optimizer - more memory efficient than Adam.

PiperOrigin-RevId: 181767655

rsepassi/tensor2tensor,rsepassi/tensor2tensor,vthorsteinsson/tensor2tensor,tensorflow/tensor2tensor,vthorsteinsson/tensor2tensor,tensorflow/tensor2tensor,vthorsteinsson/tensor2tensor,tensorflow/tensor2tensor,rsepassi/tensor2tensor,rsepassi/tensor2tensor,tensorflow/tensor2tensor,vthorsteinsson/tensor2tensor,tensorflow/tensor2tensor