DistributedShampoo sharded mode in PJIT/SPMD Transformer model.
DistributedShampoo sharded mode in PJIT/SPMD Transformer model.

PiperOrigin-RevId: 419476647

tensorflow/lingvo,tensorflow/lingvo,tensorflow/lingvo,tensorflow/lingvo