Add delta-orthogonal init that allows removal of skip-connection/normalization layers for image classification models when trained with K-FAC and Shampoo.
Add delta-orthogonal init that allows removal of skip-connection/normalization layers for image classification models when trained with K-FAC and Shampoo.

PiperOrigin-RevId: 416453639

tensorflow/lingvo,tensorflow/lingvo,tensorflow/lingvo,tensorflow/lingvo