Change parsing heaps caching strategy
Change parsing heaps caching strategy

Summary:
Before this change, we had two caches: an AST cache which was used only by
readers and an ALoc table cache used by both readers and mutators.

Remember that readers want the committed data while a mutator is writing the new
data. That is, for a given file key, readers and mutators might expect to see
different data.

The SharedMem.WithCache stuff deals with this as follows:
* oldifying a key removes its cache entry
* get_old queries bypass the cache
* reviving a key removes its cache entry

Before this diff, we used the exact same logic for the AST and ALoc table
caches. This works, but it's unfortunate that `get_old` queries bypass the
cache.

This diff separates the caches so that readers and mutators never share a cache.
Because mutators don't share a cache, it's safe for readers to continue to cache
keys which are oldified. Instead of invalidating on oldify, we invalidate on
commit.

Mutator caches are interesting. We currently only cache aloc tables to provide
the `loc_of_aloc` API used only for streaming errors. This API might look for
aloc tables for a given file repeatedly, so it is useful to cache these lookups.

Instead of trying to persist the mutator caches across rechecks, this diff just
clears the cache at the end of each recheck. My thinking is that it's probably
not worth the memory use to keep the cached values around longer. We could
certainly explore with alternate schemes that keep cached data alive across
rechecks.

This diff is mostly motivated by future changes which will fundamentally change
oldify/revive. Separating caching concerns from oldify/revive makes those future
changes easier.

Reviewed By: jbrown215

Differential Revision: D33444933

fbshipit-source-id: 13cc4163421404c24eb2bfa901a3b7ff2a9067a5

mroch/flow,facebook/flow,facebook/flow,mroch/flow,mroch/flow,facebook/flow,facebook/flow,facebook/flow,facebook/flow,facebook/flow,mroch/flow,mroch/flow,mroch/flow,mroch/flow