Avoid a potential shutdown race.
Avoid a potential shutdown race.

Testing with a research tasking layer that behaves slightly differently
than the FIFO one revealed a race in the shutdown logic of the pthreads
threading layer.  One thread entered threadlayer_exit() and traversed
the thread list, cancelling and joining with all known pthreads, and
then it deleted the key for the thread private data.  At this point
another thread, which had previously been created but had never been
selected to run by the kernel, entered pthread_func(), put itself on the
thread list, and called the tasking layer's begin-thread function.  This
then eventually called back to threadlayer_set_thread_private_data() and
tried to use that thread private data key, which attempt failed because
the key no longer existed.  An internal error resulted.

To fix this, we introduce an "exiting" flag.  The logic associated with
this flag ensures that at threadlayer exit time all threads, whether
running or created-but-not-yet-run, either put themselves on the thread
list and so get canceled and joined with, or do not put themselves on
the list and instead exit without need of cancellation and without
referencing the thread private data key.


git-svn-id: 88467cb1fb04b8a755be7e1ee1026be4190196ef@18433 3a8e244f-b0f2-452b-bcba-4c88e055c3ca

CoryMcCartan/chapel,sungeunchoi/chapel,sungeunchoi/chapel,chizarlicious/chapel,sungeunchoi/chapel,CoryMcCartan/chapel,hildeth/chapel,sungeunchoi/chapel,hildeth/chapel,hildeth/chapel,sungeunchoi/chapel,hildeth/chapel,sungeunchoi/chapel,chizarlicious/chapel,CoryMcCartan/chapel,CoryMcCartan/chapel,CoryMcCartan/chapel,sungeunchoi/chapel,hildeth/chapel,hildeth/chapel,chizarlicious/chapel,hildeth/chapel,chizarlicious/chapel,CoryMcCartan/chapel,CoryMcCartan/chapel,chizarlicious/chapel,sungeunchoi/chapel,chizarlicious/chapel,chizarlicious/chapel