runtime: fix 32 bit build.
runtime: fix 32 bit build.

int(maxMem) is negative on 32 bits.  Need to use
unsigned arithmetic.

TBR=bradfitz
CC=golang-codereviews
https://codereview.appspot.com/121000045

tv42/old-go,tv42/old-go,tv42/old-go,tv42/old-go,tv42/old-go,tv42/old-go,tv42/old-go