avoid out-of-range computation
avoid out-of-range computation

The results of the mulipliations can be larger than INT32_MAX.

Change-Id: Idc27e55c4be86b74203cabedd2c506e7762db990

GrokImageCompression/aom,GrokImageCompression/aom,GrokImageCompression/aom,GrokImageCompression/aom,GrokImageCompression/aom,GrokImageCompression/aom