prober: Make prober.Done() more consistent
prober: Make prober.Done() more consistent

Prior to this commit, when the main thread (doing API serving) starts to
shut down, it would call prober.Stop(), which would launch another
goroutine that attempted to proxy the pinger and prober's finish result
into a freshly returned socket. The main thread would wait on the
returned socket. However, multiple threads ended up racing to read from
the 'p.done' socket.

This commit splits it out so that the p.Pinger().Stop() function is
called, after which the pinger is finished; then it closes the p.stop
channel, signalling to the HTTP prober goroutine to finish. Once that
goroutine finishes, it signals the new p.proberExited channel, which is
listened from the thread running the p.Stop(). Finally, it can signal
the Done() socket and return.

This fixes some data race issues on shutdown, and simplifies the
goroutine model in cilium-health.

Signed-off-by: Joe Stringer <16a9a54ddf4259952e3c118c763138e83693d7fd@covalent.io>

tgraf/cilium,tgraf/cilium,eloycoto/cilium,cilium-team/cilium,tgraf/cilium,tklauser/cilium,tgraf/cilium,tklauser/cilium,scanf/cilium,scanf/cilium,scanf/cilium,michi-covalent/cilium,michi-covalent/cilium,michi-covalent/cilium,cilium/cilium,scanf/cilium,tgraf/cilium,tklauser/cilium,cilium/cilium,scanf/cilium,eloycoto/cilium,cilium-team/cilium,eloycoto/cilium,cilium/cilium,eloycoto/cilium,eloycoto/cilium,eloycoto/cilium,cilium/cilium,scanf/cilium,tklauser/cilium,tgraf/cilium,scanf/cilium,tklauser/cilium,eloycoto/cilium,michi-covalent/cilium,cilium/cilium,michi-covalent/cilium