Fix scheduler thread wakeup bug (#2926)
Fix scheduler thread wakeup bug (#2926)

Bug summary:

For platforms such as Linux that do not define the CPP macro
USE_SCHEDULER_SCALING_PTHREADS, it's possible for a scheduler
thread to go to sleep, wake up, and then get stuck in the
bottom half of `suspend_scheduler()` because the boolean
`scheduler_count_changing` is always `true`

Bug fix:

Maintain the invariant mentioned in the comments at the top of
`suspend_scheduler()`: when USE_SCHEDULER_SCALING_PTHREADS is
not defined, then the `scheduler_count_changing` flag must be
`false` to "unlock" the critical section.  This patch adds code
to unlock prior to a `return` near the top of
`suspend_scheduler()`.

Background:

We've seen intermittent problems with Pony runtime shutdown, for example,
this CircleCI test failure for ponyc,
https://circleci.com/gh/ponylang/ponyc/19395, where the runtime doesn't
shut down completely after the unit tests have finished executing, and
the test is killed due to 10 minutes of stdout/stderr inactivity.

On an aarch64 platform, I managed to catch a `stdlib` unit test
process while stuck after running the 303 unit tests.  The process
was consuming approx. 100% CPU time (1 CPU's/thread's worth) but
hadn't exited after 20+ minutes of runtime.  With LLDB attached, I
witnessed one of the 4 scheduler threads stuck in
`suspend_scheduler()`, busy looping through the `while (true)` loop
at the bottom of the function.  The value of `scheduler_count_changing`
was always true.  I set a watchpoint to catch when that value
might be set to false by some other thread, but the watchpoint never
tripped: the other three schedulers were usually asleep in
`ponyint_cpu_core_pause()` while attempting to `steal()` work.

I checked the other functions in scheduler.c that also change
`scheduler_count_changing`'s value but didn't find a similar
flow control error of forgetting to unlock when locked.
Reviewers: please triple-check.  ^_^
cquinn/ponyc,ponylang/ponyc,dipinhora/ponyc,CausalityLtd/ponyc,cquinn/ponyc,sgebbie/ponyc,jemc/ponyc,CausalityLtd/ponyc,dipinhora/ponyc,CausalityLtd/ponyc,Theodus/ponyc,dipinhora/ponyc,kulibali/ponyc,sgebbie/ponyc,Theodus/ponyc,sgebbie/ponyc,ponylang/ponyc,jemc/ponyc,sgebbie/ponyc,kulibali/ponyc,Theodus/ponyc,jemc/ponyc,kulibali/ponyc,kulibali/ponyc,ponylang/ponyc,sgebbie/ponyc,cquinn/ponyc,Theodus/ponyc,Theodus/ponyc,cquinn/ponyc