tabletserver: fix race condition
tabletserver: fix race condition

We discovered an interesting race condition between Begin
and StopServer:

* Begin gets called: reserves a request, but takes time to
perform the next step, state is SERVING.
* StopService gets called: transitions state to SHUTTING_DOWN.
* StopService calls WaitForTx: the pool is empty. So, it
proceeds to perform other shutdown operations.
* Begin opens a transaction, because it was given permission.
The pool is not empty any more.
* StopService calls txpool.Close, which gets angry because
no transactions should be in it.

One possible solution I considered was to reserve an entry
in the transaction pool during startRequest. Then Begin would
later fill it out with the connection obtained after BEGIN
succeeded.  This would eliminate the race, but it also
introduces weird dependencies between TabletServer and TxPool.
It would also require Begin to un-reserve if the BEGIN failed.
Overall, I felt the above approach would lead to unreadable code.

So, I went with the following approach:
* Introduce a new 'begins' waitgroup that counts number of Begin
calls in flight.
* Increment & Decrement it in startRequest and endRequest.
* Make those who want to stop query service wait for begins to
be zero before waiting on tx pool to go empty.

So, if there are any Begins in flight when we go into shut down,
we wait for them to complete. As they complete, they all reserve
a connection in the tx pool. Once the Begins are done, no new begins
are allowed, and those that completed all have created transactions.
So, at this point, it's safe to wait for tx pool to go empty.

This is not really testable. So, we have to rely on the above proof
and verification that the code was written accordingly.

HubSpot/vitess,tinyspeck/vitess,AndyDiamondstein/vitess,tirsen/vitess,dumbunny/vitess,aaijazi/vitess,dcadevil/vitess,sougou/vitess,pivanof/vitess,mapbased/vitess,mahak/vitess,yaoshengzhe/vitess,vitessio/vitess,mahak/vitess,vitessio/vitess,rnavarro/vitess,yaoshengzhe/vitess,rnavarro/vitess,mapbased/vitess,rnavarro/vitess,alainjobart/vitess,AndyDiamondstein/vitess,tirsen/vitess,erzel/vitess,mahak/vitess,dcadevil/vitess,vitessio/vitess,dcadevil/vitess,enisoc/vitess,AndyDiamondstein/vitess,davygeek/vitess,dcadevil/vitess,dumbunny/vitess,mahak/vitess,tirsen/vitess,mapbased/vitess,erzel/vitess,tirsen/vitess,applift/vitess,erzel/vitess,alainjobart/vitess,guokeno0/vitess,applift/vitess,alainjobart/vitess,applift/vitess,mattharden/vitess,alainjobart/vitess,tjyang/vitess,dumbunny/vitess,enisoc/vitess,dumbunny/vitess,HubSpot/vitess,sougou/vitess,AndyDiamondstein/vitess,HubSpot/vitess,mapbased/vitess,mahak/vitess,mattharden/vitess,dumbunny/vitess,tinyspeck/vitess,davygeek/vitess,alainjobart/vitess,guokeno0/vitess,pivanof/vitess,applift/vitess,alainjobart/vitess,alainjobart/vitess,sougou/vitess,tjyang/vitess,guokeno0/vitess,mattharden/vitess,rnavarro/vitess,tjyang/vitess,mapbased/vitess,tinyspeck/vitess,tjyang/vitess,mapbased/vitess,dumbunny/vitess,mattharden/vitess,erzel/vitess,mapbased/vitess,vitessio/vitess,applift/vitess,davygeek/vitess,yaoshengzhe/vitess,dcadevil/vitess,dcadevil/vitess,dumbunny/vitess,pivanof/vitess,erzel/vitess,sougou/vitess,tinyspeck/vitess,aaijazi/vitess,tjyang/vitess,mattharden/vitess,alainjobart/vitess,yaoshengzhe/vitess,aaijazi/vitess,dumbunny/vitess,applift/vitess,sougou/vitess,rnavarro/vitess,vitessio/vitess,mattharden/vitess,tirsen/vitess,yaoshengzhe/vitess,tjyang/vitess,AndyDiamondstein/vitess,tinyspeck/vitess,mattharden/vitess,tjyang/vitess,aaijazi/vitess,aaijazi/vitess,applift/vitess,tirsen/vitess,rnavarro/vitess,pivanof/vitess,AndyDiamondstein/vitess,tinyspeck/vitess,mahak/vitess,guokeno0/vitess,mattharden/vitess,davygeek/vitess,AndyDiamondstein/vitess,sougou/vitess,yaoshengzhe/vitess,HubSpot/vitess,erzel/vitess,enisoc/vitess,guokeno0/vitess,tjyang/vitess,guokeno0/vitess,yaoshengzhe/vitess,mapbased/vitess,yaoshengzhe/vitess,mapbased/vitess,erzel/vitess,pivanof/vitess,davygeek/vitess,applift/vitess,pivanof/vitess,aaijazi/vitess,vitessio/vitess,tirsen/vitess,enisoc/vitess,sougou/vitess,guokeno0/vitess,aaijazi/vitess,mahak/vitess,erzel/vitess,enisoc/vitess,davygeek/vitess,rnavarro/vitess,erzel/vitess,guokeno0/vitess,mattharden/vitess,erzel/vitess,tirsen/vitess,tjyang/vitess,tirsen/vitess,enisoc/vitess,enisoc/vitess,davygeek/vitess,rnavarro/vitess,dumbunny/vitess,vitessio/vitess,dcadevil/vitess,pivanof/vitess,mahak/vitess,mapbased/vitess,dumbunny/vitess,rnavarro/vitess,HubSpot/vitess,applift/vitess,tinyspeck/vitess,pivanof/vitess,sougou/vitess,applift/vitess,yaoshengzhe/vitess,AndyDiamondstein/vitess,aaijazi/vitess,HubSpot/vitess,HubSpot/vitess,mattharden/vitess,AndyDiamondstein/vitess,aaijazi/vitess,HubSpot/vitess,pivanof/vitess,guokeno0/vitess,vitessio/vitess