Supports Rotary position embedding in Decoder Self attention.
Supports Rotary position embedding in Decoder Self attention.

PiperOrigin-RevId: 454533757

tensorflow/lingvo,tensorflow/lingvo,tensorflow/lingvo,tensorflow/lingvo