Attempt to fix RainfallHistorgramSink
Attempt to fix RainfallHistorgramSink

recording a latency was using the RainfallHistorgramSink facade to record the latencies into a static HistogramHolder, which is defined per thread using a ThreadLocal.
because the thread operations are taken from a ThreadPool, the first time a Thread is recording, and no HistogramHolder exist yet, it is created, then reused by all other operations executed by that Thread, regardless of the result.

e.g.
histograms.get(result).record(latency)
First time it calls:
hh = new HistogramHolder<E>(factory.createHistogram());
Then every other times, it will reuse that histogram holder
So let’s say, that the first time, for "Thread 1” of the thread pool,  the result = MISS, it will get the RainfallHistogramSink that will create the HistogramHolder
Second time, for Thread 1, say the result is MISS again, it will get the same RainfallHistogramSink that already has the HistogramHolder instantiated so it will record correctly.
Then Third time for Thread 1, say now the result is PUT, it will get another RainfallHistogramSink, and when getting the HistogramHolder from the ThreadLocal, it will get the one holding values for the MISS, so it will record the latency there.

In the end, the first results to have called a thread will ‘own’ the HistogramHolder and you will get erroneous Histograms because they will be an addition of any result from the operation executed by that Thread.

Proposed fix:
move the CHM down into the RainfallHistogramSink.
the ThreadLocal of the RainfallHistogramSink will have the CHM and record each result independently for every thread
when fetching the results at the end, all actives HistogramHolders (from all threads) are aggregated for a specific Result.

aurbroszniowski/Rainfall-ehcache