The current attention computes compatibilities between single query, key, and value positions. This CL extends it to computing them between windows of queries, keys, and values. It's like a combination of convolution and attention. Does not change defaults.
The current attention computes compatibilities between single query, key, and value positions. This CL extends it to computing them between windows of queries, keys, and values. It's like a combination of convolution and attention. Does not change defaults.

PiperOrigin-RevId: 165405437

tensorflow/tensor2tensor,tensorflow/tensor2tensor,rsepassi/tensor2tensor,vthorsteinsson/tensor2tensor,tensorflow/tensor2tensor,vthorsteinsson/tensor2tensor,tensorflow/tensor2tensor,waterblue13/tensor2tensor,rsepassi/tensor2tensor,tensorflow/tensor2tensor,vthorsteinsson/tensor2tensor,vthorsteinsson/tensor2tensor,rsepassi/tensor2tensor,waterblue13/tensor2tensor,waterblue13/tensor2tensor,rsepassi/tensor2tensor