Schedule vbucket replications in small batches
Schedule vbucket replications in small batches

Previously, xdcr manager would trigger couch replications for all
active vbuckets simultaneously which was causing performance to
suffer. The new approach, instead, schedules replications for only a
small number of vbuckets (32 by default and configurable through an
environment variable called MAX_CONCURRENT_REPS_PER_DOC) at any given
time.

This change set only supports xdcr continuous mode replication. We
start by scheduling the specified number of vbuckets (in couch mode
non-continuous) and periodicaly monitor them. As one or more of them
are done, new vbuckets are scheduled. After we've run through all of
them in this manner, we repeat the process.

An alternate approach is likely better where we can assign a time
slot for each vbucket instead of letting it run through completion
before scheduling another. Implementing this approach would be
greatly simplified if the Couch replicator API supported pause and
resume functions. Until then, we can stick to the current approach.

Also, I found it harded to break this change set into smaller pieces.
If reviewing seems hard or if there's a better way to organize stuff,
please let me know.

Change-Id: Ifc905becb41296e68ee66dbdc73b912bc055d15a
Reviewed-on: http://review.couchbase.org/13663
Tested-by: Srinivas Vadlamani <4036f57732a648e71da3ac2c829c8239a16a4c5d@couchbase.com>
Reviewed-by: Aliaksey Kandratsenka <340b8e09ca65cd3fc686427fcfed17e87eaf61e2@gmail.com>
Tested-by: Aliaksey Kandratsenka <340b8e09ca65cd3fc686427fcfed17e87eaf61e2@gmail.com>

nimishzynga/ns_server,membase/ns_server,mschoch/ns_server,mschoch/ns_server,ceejatec/ns_server,daverigby/ns_server,t3rm1n4l/ns_server,pavel-blagodov/ns_server,nimishzynga/ns_server,vzasade/ns_server,nimishzynga/ns_server,vzasade/ns_server,daverigby/ns_server,membase/ns_server,ceejatec/ns_server,t3rm1n4l/ns_server,ceejatec/ns_server,mschoch/ns_server,membase/ns_server,pavel-blagodov/ns_server,membase/ns_server,mschoch/ns_server,mschoch/ns_server,pavel-blagodov/ns_server,ceejatec/ns_server,ceejatec/ns_server,daverigby/ns_server,nimishzynga/ns_server,t3rm1n4l/ns_server,membase/ns_server,pavel-blagodov/ns_server,t3rm1n4l/ns_server,t3rm1n4l/ns_server,daverigby/ns_server,vzasade/ns_server,nimishzynga/ns_server,daverigby/ns_server,nimishzynga/ns_server,vzasade/ns_server,mschoch/ns_server,membase/ns_server,ceejatec/ns_server