database: memtables pending flushes tell us nothing
database: memtables pending flushes tell us nothing

We have two counters that tracks how many memtable flushes are in progress, and
how much memory are they pinning.

The problem is, after we have revamped the code to limit the amount of flushes
in progress, those counters became useless: as they live inside the semaphore
side, they will only be incremented once we have past the semaphore.

One wouldn't notice if working with CPU-bound problems, where memtables don't
pile. But as soon as they do, those counters will always show the same numbers:
the depth of the semaphore, which doesn't mean much. The problem is poised to
become much worse: once we enable write behind in full and set the semaphore's
depth to one, that's the number we'll see here all the time.

The fix is to move the counters outside the semaphore, which will bring back its
old semantics.

Signed-off-by: Glauber Costa <7ba9037b0dd006f9408bd45b94707becd5cd4e87@scylladb.com>
Message-Id: <c5ae6903e170f3f356cdda7ed78a4c9ba8d5f024.1471370504.git.7ba9037b0dd006f9408bd45b94707becd5cd4e87@scylladb.com>

avikivity/scylla,scylladb/scylla,avikivity/scylla,scylladb/scylla,raphaelsc/scylla,duarten/scylla,raphaelsc/scylla,duarten/scylla,raphaelsc/scylla,scylladb/scylla,duarten/scylla,scylladb/scylla,avikivity/scylla