Avoid storing optimizer variables in multiple replicas. initialize it on the PS device.
Avoid storing optimizer variables in multiple replicas. initialize it on the PS device.

rossumai/keras-multi-gpu,rossumai/keras-multi-gpu