runtime: adjust huge page flags only on huge page granularity
runtime: adjust huge page flags only on huge page granularity

This fixes an issue where the runtime panics with "out of memory" or
"cannot allocate memory" even though there's ample memory by reducing
the number of memory mappings created by the memory allocator.

Commit 7e1b61c worked around issue #8832 where Linux's transparent
huge page support could dramatically increase the RSS of a Go process
by setting the MADV_NOHUGEPAGE flag on any regions of pages released
to the OS with MADV_DONTNEED. This had the side effect of also
increasing the number of VMAs (memory mappings) in a Go address space
because a separate VMA is needed for every region of the virtual
address space with different flags. Unfortunately, by default, Linux
limits the number of VMAs in an address space to 65530, and a large
heap can quickly reach this limit when the runtime starts scavenging
memory.

This commit dramatically reduces the number of VMAs. It does this
primarily by only adjusting the huge page flag at huge page
granularity. With this change, on amd64, even a pessimal heap that
alternates between MADV_NOHUGEPAGE and MADV_HUGEPAGE must reach 128GB
to reach the VMA limit. Because of this rounding to huge page
granularity, this change is also careful to leave large used and
unused regions huge page-enabled.

This change reduces the maximum number of VMAs during the runtime
benchmarks with GODEBUG=scavenge=1 from 692 to 49.

Fixes #12233.

Change-Id: Ic397776d042f20d53783a1cacf122e2e2db00584
Reviewed-on: https://go-review.googlesource.com/15191
Reviewed-by: Keith Randall <8c99c3a9284e493be632950b84cd789d08ed3e9d@golang.org>

theif519/go,derekmarcotte/go,jacobsa/go,rachtsingh/gogojuice,Cofyc/go,mwhudson/go,vanloswang/go,gdey/go,theif519/go,josharian/go,vanloswang/go,ganboing/go-esx,nawawi/go,jacobsa/go,binarycrusader/go,roth1002/go,suncycheng/go,fjballest/golang,derekmarcotte/go,golang/go,sndnvaps/go,theif519/go,ganboing/go-esx,deft-code/go,mwhudson/go,gdey/go,ImJasonH/go,Endika/go,Cofyc/go,fjballest/golang,odeke-em/go,kim-racktop/go,roth1002/go,vsekhar/elastic-go,4ad/go,momchil-velikov/go,linux-on-ibm-z/go,CAFxX/go,dterei/go,4ad/go,odeke-em/go,Cofyc/go,mwhudson/go,jacobsa/go,Cofyc/go,sndnvaps/go,Samurais/go,kim-racktop/go,4ad/go,Jordanzuo/go,CAFxX/go,mwhudson/go,derekmarcotte/go,sndnvaps/go,deft-code/go,deft-code/go,derekmarcotte/go,vanloswang/go,Jordanzuo/go,gdey/go,codestation/go,theif519/go,cyberroadie/go,momchil-velikov/go,golang/go,CAFxX/go,cloudflare/go,rachtsingh/gogojuice,momchil-velikov/go,theif519/go,Samurais/go,ImJasonH/go,jacobsa/go,suncycheng/go,dterei/go,akutz/go,theif519/go,kim-racktop/go,deft-code/go,linux-on-ibm-z/go,odeke-em/go,4ad/go,malvira/go,kim-racktop/go,fjballest/golang,sndnvaps/go,codestation/go,goodlang/good,kim-racktop/go,akutz/go,fjballest/golang,Endika/go,rachtsingh/gogojuice,ImJasonH/go,goodlang/good,ImJasonH/go,codestation/go,malvira/go,binarycrusader/go,nawawi/go,derekmarcotte/go,roth1002/go,polyverse-security/go,fjballest/golang,sndnvaps/go,goodlang/good,cyberroadie/go,nawawi/go,4ad/go,malvira/go,sndnvaps/go,cloudflare/go,derekmarcotte/go,josharian/go,Endika/go,cyberroadie/go,josharian/go,miolini/go,suncycheng/go,gdey/go,golang/go,mwhudson/go,Jordanzuo/go,Endika/go,linux-on-ibm-z/go,deft-code/go,binarycrusader/go,polyverse-security/go,josharian/go,cloudflare/go,dterei/go,akutz/go,ganboing/go-esx,deft-code/go,rachtsingh/gogojuice,binarycrusader/go,polyverse-security/go,rachtsingh/gogojuice,malvira/go,momchil-velikov/go,dterei/go,linux-on-ibm-z/go,suncycheng/go,Samurais/go,codestation/go,Cofyc/go,miolini/go,CAFxX/go,cyberroadie/go,derekmarcotte/go,sndnvaps/go,miolini/go,cloudflare/go,suncycheng/go,momchil-velikov/go,gdey/go,akutz/go,nawawi/go,binarycrusader/go,vanloswang/go,4ad/go,rachtsingh/gogojuice,binarycrusader/go,binarycrusader/go,jacobsa/go,polyverse-security/go,ImJasonH/go,vanloswang/go,cyberroadie/go,odeke-em/go,josharian/go,Jordanzuo/go,linux-on-ibm-z/go,christopher-henderson/Go,momchil-velikov/go,linux-on-ibm-z/go,miolini/go,vsekhar/elastic-go,dterei/go,codestation/go,vsekhar/elastic-go,ImJasonH/go,cyberroadie/go,mwhudson/go,goodlang/good,cyberroadie/go,codestation/go,momchil-velikov/go,vsekhar/elastic-go,roth1002/go,ganboing/go-esx,linux-on-ibm-z/go,polyverse-security/go,vsekhar/elastic-go,mwhudson/go,golang/go,roth1002/go,dterei/go,Endika/go,nawawi/go,Cofyc/go,4ad/go,suncycheng/go,miolini/go,kim-racktop/go,derekmarcotte/go,vsekhar/elastic-go,Samurais/go,codestation/go,Samurais/go,suncycheng/go,sndnvaps/go,kim-racktop/go,ganboing/go-esx,josharian/go,jacobsa/go,jacobsa/go,mwhudson/go,CAFxX/go,theif519/go,roth1002/go,nawawi/go,polyverse-security/go,roth1002/go,Samurais/go,malvira/go,jacobsa/go,Endika/go,cyberroadie/go,dterei/go,dterei/go,vanloswang/go,ImJasonH/go,theif519/go,4ad/go,Jordanzuo/go,fjballest/golang,Jordanzuo/go,cloudflare/go,golang/go,odeke-em/go,ganboing/go-esx,akutz/go,miolini/go,gdey/go,fjballest/golang,suncycheng/go,CAFxX/go,CAFxX/go,rachtsingh/gogojuice,vsekhar/elastic-go,golang/go,cloudflare/go,codestation/go,Samurais/go,ganboing/go-esx,malvira/go,golang/go,ganboing/go-esx,akutz/go,nawawi/go,Jordanzuo/go,Cofyc/go,polyverse-security/go,goodlang/good,Endika/go,vanloswang/go,Samurais/go,miolini/go,malvira/go,miolini/go,odeke-em/go,kim-racktop/go,malvira/go,josharian/go,Jordanzuo/go,vanloswang/go,deft-code/go,nawawi/go,Endika/go,josharian/go,roth1002/go,gdey/go,gdey/go,vsekhar/elastic-go,golang/go,polyverse-security/go,goodlang/good,akutz/go,linux-on-ibm-z/go,momchil-velikov/go,cloudflare/go,odeke-em/go,goodlang/good,rachtsingh/gogojuice,akutz/go,cloudflare/go,odeke-em/go,ImJasonH/go,deft-code/go,CAFxX/go,binarycrusader/go,Cofyc/go,goodlang/good,fjballest/golang