Bug fix, no access to targets during decoding. Move to correct place
Bug fix, no access to targets during decoding. Move to correct place

PiperOrigin-RevId: 166791783

rsepassi/tensor2tensor,vthorsteinsson/tensor2tensor,vthorsteinsson/tensor2tensor,tensorflow/tensor2tensor,tensorflow/tensor2tensor,rsepassi/tensor2tensor,tensorflow/tensor2tensor,tensorflow/tensor2tensor,rsepassi/tensor2tensor,waterblue13/tensor2tensor,tensorflow/tensor2tensor,vthorsteinsson/tensor2tensor,waterblue13/tensor2tensor,waterblue13/tensor2tensor,vthorsteinsson/tensor2tensor,rsepassi/tensor2tensor