logalloc: allow region group reclaimer to specify a soft limit
logalloc: allow region group reclaimer to specify a soft limit

The region_group_reclaimer will let us know every time we are over the
limit we have specified for memory usage.

However, For some applications, we would be interested in knowing about
memory build up earlier, so we can start doing something about it before
we reach that condition.

This patch introduce soft limit notifications for the
region_group_reclaimer. After this patch is applied, start_reclaim() is
called earlier, and stop_reclaim() later, after the soft condition is
abated.

There are methods that allow one to easily test if the pressure
condition is a soft limit condition or a hard, threshold condition and
act accordingly. Whether to act on both conditions or just one of them
is up to the application.

Signed-off-by: Glauber Costa <7ba9037b0dd006f9408bd45b94707becd5cd4e87@scylladb.com>

avikivity/scylla,raphaelsc/scylla,scylladb/scylla,avikivity/scylla,scylladb/scylla,scylladb/scylla,raphaelsc/scylla,raphaelsc/scylla,duarten/scylla,avikivity/scylla,duarten/scylla,scylladb/scylla,duarten/scylla