Add a gin-configurable offset argument to adafactor_decay_rate_pow()
Add a gin-configurable offset argument to adafactor_decay_rate_pow()

The intended use case is that in fine-tuning, offset can be set to the starting step-number of the fine-tuning phase.
This lets Adafactor adjust rapidly in the case where the losses are much larger or much smaller than the losses during pre-training, (e.g. smaller batch size, but gin_configuring loss_denominator to match pre-training).  Crucially, it makes the update-clipping work properly.  Caveat: you may need a different learning rate during fine-tuning.

PiperOrigin-RevId: 338572875

tensorflow/mesh,tensorflow/mesh