Fix a race condition in distributed training.
Fix a race condition in distributed training.

Each worker monitors the iteration's `is_over` boolean variable to know when to stop training and prepare for the next iteration.

Previously, each worker would assign its version of the training status to of `is_over` at each step without first reading it, which could end up with `True` values being set to `False` by slower workers.

With this change, workers can only assign the `is_over` variable to True. This way the fastest worker always determines when training is over.

PiperOrigin-RevId: 226491134

tensorflow/adanet,tensorflow/adanet