Major threading optimizations:  1. Use only N-1 worker threads while the master plays the role     of the Nth worker, where N is the number of cores.     This 1:1 mapping of threads to cores gives much better perf     esp. for not-very-large GEMMs and esp. on Android.  2. Implement waiting by actually busy-waiting for a little while     before eventually falling back to passive waiting. That     ensures that we wake up quickly from short naps, which helps     with not-very-large GEMMs esp. on Android.
Major threading optimizations:
 1. Use only N-1 worker threads while the master plays the role
    of the Nth worker, where N is the number of cores.
    This 1:1 mapping of threads to cores gives much better perf
    esp. for not-very-large GEMMs and esp. on Android.
 2. Implement waiting by actually busy-waiting for a little while
    before eventually falling back to passive waiting. That
    ensures that we wake up quickly from short naps, which helps
    with not-very-large GEMMs esp. on Android.

maciekcc/gemmlowp,google/gemmlowp,petewarden/gemmlowp,google/gemmlowp,dave-andersen/gemmlowp,petewarden/gemmlowp,miaowang14/gemmlowp,petewarden/gemmlowp,efeguney/gemmlowp,google/gemmlowp,maciekcc/gemmlowp,miaowang14/gemmlowp,efeguney/gemmlowp,dave-andersen/gemmlowp,efeguney/gemmlowp,google/gemmlowp,dave-andersen/gemmlowp,miaowang14/gemmlowp,petewarden/gemmlowp,maciekcc/gemmlowp