api: storage_service: force_keyspace_compaction: compact one table at a time
api: storage_service: force_keyspace_compaction: compact one table at a time

To make major compaction more resilient to low-
disk space conditions, 342bfbd65ab49b043f08ca5694d813a601a05f27
sorted the tables based on their live disk space used.

However, each shard still makes progress in its own pace.

This change serializes major compaction between tables
so we still compact in parallel on all shards, but one
(distributed) table at a time.

As a follow-up, we can consider serializing even at the single shard
level when disk space is critically low, so we can't even risk
parallel compaction across all shards.

Refs scylladb/scylla-dtest#2653

Test: unit(dev)

Signed-off-by: Benny Halevy <23c997c90eb8537635e9392d48e7af1bf1b2ca22@scylladb.com>
Message-Id: <20220313153814.2203660-1-23c997c90eb8537635e9392d48e7af1bf1b2ca22@scylladb.com>

scylladb/scylla,scylladb/scylla,scylladb/scylla,scylladb/scylla