Complete wiring up Shampoo for model parallel training for really large models.
Complete wiring up Shampoo for model parallel training for really large models.

PiperOrigin-RevId: 418902736

google-research/google-research,google-research/google-research,google-research/google-research,google-research/google-research,google-research/google-research,google-research/google-research,google-research/google-research,google-research/google-research,google-research/google-research,google-research/google-research,google-research/google-research