Fix deadlock in clock.Timer
Fix deadlock in clock.Timer

The following deadlock occurs in clock.Timer code from time to time:

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
github.com/WatchBeam/clock.(*mockTimer).Stop(0xc42000e140, 0x0)
        /Users/pbor/src/github.com/WatchBeam/clock/timer.go:78 +0xb6
main.main()
        /tmp/repro.go:14 +0x1aa

goroutine 3122 [semacquire]:
sync.runtime_Semacquire(0xc42000e154)
        /usr/local/Cellar/go/1.7.3/libexec/src/runtime/sema.go:47 +0x30
sync.(*Mutex).Lock(0xc42000e150)
        /usr/local/Cellar/go/1.7.3/libexec/src/sync/mutex.go:85 +0xd0
github.com/WatchBeam/clock.(*mockTimer).setInactive(0xc42000e140)
        /Users/pbor/src/github.com/WatchBeam/clock/timer.go:28 +0x3a
github.com/WatchBeam/clock.(*mockTimer).wait(0xc42000e140)
        /Users/pbor/src/github.com/WatchBeam/clock/timer.go:46 +0x1a2
created by github.com/WatchBeam/clock.(*mockTimer).Reset
        /Users/pbor/src/github.com/WatchBeam/clock/timer.go:66 +0x124

The following small program reproduces the problem in a fraction of a
second:

package main

import "fmt"
import "time"
import "github.com/WatchBeam/clock"

func main() {
	now := time.Date(2016, 10, 28, 10, 10, 10, 0, time.Local)
	cl := clock.NewMockClock(now)

	tmr := cl.NewTimer(0)
	for {
		fmt.Println("iteration")
		if !tmr.Stop() {
			<-tmr.Chan()
		}
		tmr.Reset(0)
	}
}

This occurs because setInactive() called from wait() cannot read from
'release' channel (it cannot proceed because of the locked mutex) and
Stop() cannot proceed because no-one is reading the channel.

This could have been fixed by using TryLock() in setInactive(), but,
unfortunately, Go's 'sync' package doesn't provide such a primitive.

So this patches changes Timer from using a Mutex for locking to using a
buffered channel, which allows to kind of simulate TryLock semantics
using 'select' statement.

Signed-off-by: Pavel Borzenkov <073b1820d4178210f16262dddeb2ba44fb7467d3@gmail.com>

WatchBeam/clock