MB-5307: optimized ebucketmigrator network delays
MB-5307: optimized ebucketmigrator network delays

It was found that Naggle algorithm delays are slowing down rebalances
quite a lot. On the other hand just naively setting TCP_NODELAY option
does not work, because ebucketmigrator is using send per tap
message. And I've found it to slow down takeovers that require sending
some data volumes.

Alternative approach of permanently having TCP_NODELAY on upstream
socket and briefly enabling TCP_NODELAY on downstream socket for
pushing of previously sent messages seems to work well. The idea is
that erlang & memcached will batch enough stuff that we'll receive as
one piece in handle_info. And then pass to process_data. Process data
will do (potentially a bunch of) send calls. If after that we don't
have full frame transmission may be delayed according to Naggle's
algorithm. So we "push" this stuff through.

Something better perhaps can be done. I.e. we could detect messages
that we need to deliver asap (most likely with ack flag set) and only
push them. But implemented behavior seems to be good enough. My
thinking is that memcached is seemingly doing 10 messages per send
syscall. Which for short messages is already imperfect. And memcached
is using TCP_NODELAY. So we should be matching their questionable
behavior. But hopefully not making it significantly worse.

Change-Id: Ibc36bd8b010f3052c6731e78db656cf2cd7369bb
Reviewed-on: http://review.couchbase.org/16781
Reviewed-by: Aliaksey Artamonau <3c875bcfb3adf2a65b2ae7686ca921e6c9433147@gmail.com>
Tested-by: Aliaksey Kandratsenka <340b8e09ca65cd3fc686427fcfed17e87eaf61e2@gmail.com>

vzasade/ns_server,pavel-blagodov/ns_server,pavel-blagodov/ns_server,daverigby/ns_server,pavel-blagodov/ns_server,vzasade/ns_server,daverigby/ns_server,ceejatec/ns_server,membase/ns_server,mschoch/ns_server,ceejatec/ns_server,vzasade/ns_server,ceejatec/ns_server,ceejatec/ns_server,nimishzynga/ns_server,t3rm1n4l/ns_server,mschoch/ns_server,membase/ns_server,membase/ns_server,pavel-blagodov/ns_server,t3rm1n4l/ns_server,t3rm1n4l/ns_server,mschoch/ns_server,t3rm1n4l/ns_server,nimishzynga/ns_server,daverigby/ns_server,ceejatec/ns_server,membase/ns_server,nimishzynga/ns_server,nimishzynga/ns_server,daverigby/ns_server,nimishzynga/ns_server,mschoch/ns_server,t3rm1n4l/ns_server,membase/ns_server,nimishzynga/ns_server,daverigby/ns_server,ceejatec/ns_server,membase/ns_server,mschoch/ns_server,vzasade/ns_server,mschoch/ns_server