Segment compact and repair commands to improve performance (#1433)
Segment compact and repair commands to improve performance (#1433)

* Add repair wrapper
* store as object
* Move log message
* Fix the rest of the logging
* Don't try to repair deleted metrics and better logging
* Try again immediately, rather than after a full run
* Actually track failed metrics
* Chunk repairs into 1 hour increments to reduce 'timeout' condition
* Better chunking and some comments on weird code blocks
* Return failed metrics correctly
* Only log chunks when they take a long time
* Spit repairs even further (30 minutes by default)
* Improve logging
* More updates to repair tool
* Track finished metrics between runs...
* Only write success when success happens
* Better warnings around slow chunks
* Multiprocessing of metrics (speeds things up)
* flake8 fixes
* Global counter to track progress
* Configurable threads and track failed metrics
* Don't require compaction
* 'uid metrics' command doesn't reliably exit, use 'uid grep metrics' instead
* Split compact and repair commands, as they seem to cause scanner problems in HBase 2.0
* Slightly better logging and default to hour chunks (better compact?)
* Actually kill subprocesses when timeout is reached
* flake8 fixes
* Allow for time chunks larger than 60, but still ensure alignment
* Simplify to list comprehension
* More threads, increasing timeouts for bigger metrics, and actually kill timed out subprocesses
* Nicer log message and fix bad setting
* Better logging

Signed-off-by: Chris Larsen <83037b211e1d109d6858769a113e269c3139ffd3@yahoo-inc.com>
OpenTSDB/opentsdb,MadDogTechnology/opentsdb,MadDogTechnology/opentsdb,OpenTSDB/opentsdb,OpenTSDB/opentsdb,MadDogTechnology/opentsdb