Added FP16SgdOptimizer
Added FP16SgdOptimizer

Summary:
Added FP16SgdOptimizer to optimizers. The optimizer updates the params using the FP16MomentumSGDUpdate and FP32MomentumSGDUpdate ops. To determine which update op to call the optimizer expects either the fp32_update flag to be set, or that the blobs are in a recognized format created by initializers.py.

These requirements can be loosened if the blob DataType can be queried in python, though I am unsure of how to do this.

It also forces FP32 updates to SpatialBN as CuDNN does not support FP32 params for SpatialBN.

Reviewed By: asaadaldien

Differential Revision: D5840806

fbshipit-source-id: 84ab8dc11a6e91a198ed72c00287f4809607079d

xzturn/caffe2,sf-wind/caffe2,sf-wind/caffe2,xzturn/caffe2,Yangqing/caffe2,sf-wind/caffe2,xzturn/caffe2,Yangqing/caffe2,pietern/caffe2,pietern/caffe2,pietern/caffe2,davinwang/caffe2,davinwang/caffe2,davinwang/caffe2,Yangqing/caffe2,sf-wind/caffe2,pietern/caffe2,caffe2/caffe2,xzturn/caffe2,sf-wind/caffe2,Yangqing/caffe2,pietern/caffe2,davinwang/caffe2,Yangqing/caffe2,davinwang/caffe2,xzturn/caffe2