compaction: Fix the 2x disk space requirement in SSTable upgrade
compaction: Fix the 2x disk space requirement in SSTable upgrade

SSTable upgrade is requiring 2x the space of input SSTables because
we aren't releasing references of the SSTables that were already
upgraded. So if we're upgrading 1TB, it means that up to 2TB may be
required for the upgrade operation to succeed.

That can be fixed by moving all input SSTables when rewrite_sstables()
asks for the set of SSTables to be compacted, so allowing their space
to be released as soon as there is no longer any ref to them.

Spotted while auditting code.

Fixes #6682.

Signed-off-by: Raphael S. Carvalho <1e2942cea40e3b4d2162dfcd9e53e41c05a84d22@scylladb.com>
Message-Id: <20200619205701.92891-1-1e2942cea40e3b4d2162dfcd9e53e41c05a84d22@scylladb.com>

scylladb/scylla,scylladb/scylla,avikivity/scylla,avikivity/scylla,scylladb/scylla,avikivity/scylla,scylladb/scylla