Fix bug in infinite positional embedding where the inputs were also transformed. Fix bug in infinite positional embedding where the angles were randomized each iteration. Also add time_bin_length feature, which helps transform='diag' be as powerful as transform='full', but taking longer to train.
Fix bug in infinite positional embedding where the inputs were also transformed.
Fix bug in infinite positional embedding where the angles were randomized each iteration.
Also add time_bin_length feature, which helps transform='diag' be as powerful as transform='full', but taking longer to train.

PiperOrigin-RevId: 300078269

google/trax,google/trax