Self-attention feed forward layer. Replaces the feed-fwd layer with a layer that does self attention across channel depth.
Self-attention feed forward layer. Replaces the feed-fwd layer with a layer that does self attention across channel depth.

PiperOrigin-RevId: 160556355

rsepassi/tensor2tensor,vthorsteinsson/tensor2tensor,waterblue13/tensor2tensor,vthorsteinsson/tensor2tensor,tensorflow/tensor2tensor,tensorflow/tensor2tensor,tensorflow/tensor2tensor,rsepassi/tensor2tensor,tensorflow/tensor2tensor,tensorflow/tensor2tensor,waterblue13/tensor2tensor,rsepassi/tensor2tensor,vthorsteinsson/tensor2tensor,waterblue13/tensor2tensor,rsepassi/tensor2tensor,vthorsteinsson/tensor2tensor