Improve test duration under Valgrind
Improve test duration under Valgrind

Compared to the parent commit, this change reduces the run time of tests
under Valgrind from 1051s (17m31) to 295s (4m55) - roughly a 72%
reduction.

This reduction comes from the teardown phase of a number of tests.
When destroying an ExecutorPool, all the taskables must be unregistered.

Prior to this change, when unregistering the last taskable from an
ExecutorPool, all threads were woken and prevented from sleeping (by
pretending there is work, numReadyTasks[idx]++) /before/ they were
marked as dead. During this time the woken threads all busy-wait for
work.

Usually this wouldn't be too problematic because the main thread would
mark them dead soon after. However, under Valgrind only one thread can
execute at a time, and it can take significant time before the main
thread is able to mark all of the ExecutorThreads as dead.

This change simply marks them dead before waking them. They check their
status, find it is "dead", and exit.

Change-Id: Ic88ccec46ac26d511a18a1370d20117af173703b
Reviewed-on: http://review.couchbase.org/78083
Tested-by: Build Bot <80754af91bfb6d1073585b046fe0a474ce868509@couchbase.com>
Reviewed-by: Dave Rigby <a09264da4832c7ff1d3bf1608a19f4b870f93750@couchbase.com>

membase/ep-engine,couchbase/ep-engine,daverigby/kv_engine,membase/ep-engine,membase/ep-engine,couchbase/ep-engine,membase/ep-engine,daverigby/kv_engine,couchbase/ep-engine,couchbase/ep-engine,daverigby/kv_engine,daverigby/kv_engine