PR#7220: fix a memory leak of backtrace buffer in systhreads code
PR#7220: fix a memory leak of backtrace buffer in systhreads code

The following reproduction code by Rob Hoes consumes all the memory of a machine:

  let _ =
   Printexc.record_backtrace true;
   let rec loop () =
     let t = Thread.create (fun () ->
       try
         raise Not_found
       with Not_found ->
         ()
     ) () in
     Thread.join t;
     loop ()
   in
   loop ()

The following is my understanding of what is happening before the current patch.

caml_thread_{enter,leave}_blocking_section follow a bracketing discipline where
- thread_enter_blocking_section will save global runtime state into
  a thread-specific storage space (the caml_thread_struct),
  including caml_backtrace_buffer and related backtrace-related globals
- thread_leave_blocking_section will restore the runtime state
  from the thread-specific storage

st_stubs.c also has a "master runtime lock" that protects the global
data of the scheduler (list of current threads, etc.). This lock must
be held at thread creation, deletion, scheduling time. It is also held
as long as a caml thread is still running, that is outside the
blocking section: it is acquired by
caml_thread_leave_blocking_section, and released by
caml_thread_enter_blocking_section.

I will call "thread-specific backtrace buffer" the value of the
thread-specific stored backtrace buffer, and "global backtrace buffer"
the global caml_backtrace_buffer value.

Upon creating a new thread, the thread-specific backtrace buffer is
set to NULL. Upon freeing a thread, it is freed if not NULL
anymore.

The memory leak happens as the following events happen in order
in your reproduction case, when a thread is created and then joined
from the "main thread":

0. thread creation: the new thread is created with its thread-specific
   buffer set to NULL

1. thread join (into the scheduler): the main thread releases control, enters the
   blocking section, storing the current global backtrace buffer as
   its thread-specific buffer.

2. thread join (out of the scheduler): the new thread gains control,
   leaves the blocking section, restoring its current thread-specific
   buffer (NULL) into the global backtrace buffer.

3. An exception is raised with backtrace enabled; the runtime tries to
   collect the backtrace into the global backtrace buffer, notices that it is NULL,
   and malloc() a new backtrace buffer in it. The thread-specific buffer value is
   *not* changed, it is still NULL.

4. When a thread terminate, caml_thread_stop() is called (see
   caml_thread_start declaration for how this is
   implemented). caml_thread_stop calls caml_thread_remove_info which
   cleanups the thread-local structure; but the thread-local backtrace
   buffer of the new thread is still NULL, so nothing happens. Control
   is given back to the scheduler by releasing the master runtime
   lock, *without* calling caml_thread_enter_blocking_section (this
   would be invalid as the thread-specific storage does not exist, and
   also we must leave in the same state we were when we started the
   thread, and that is just after calling
   caml_thread_leave_blocking_section from the main thread).

The memory leak happens between steps (3) and (4): a new backtrace
buffer has been allocated by the backtrace code, but the thread-local
structure does not know about it so it is not properly cleaned-up when
stopping the thread.

The patch changes the caml_thread_stop() code to save the current
state in the thread-local storage at the very beginning of
caml_thread_stop. This means that the cleanup logic in
caml_thread_remove_info will see accurate values and thus deallocate
the new backtrace buffer at step (4).

yunxing/ocaml,yunxing/ocaml,yunxing/ocaml,msprotz/ocaml,eliberis/ocaml,eliberis/ocaml,gerdstolpmann/ocaml,msprotz/ocaml,yunxing/ocaml,eliberis/ocaml,gerdstolpmann/ocaml,yunxing/ocaml,eliberis/ocaml,msprotz/ocaml,msprotz/ocaml,eliberis/ocaml,gerdstolpmann/ocaml,msprotz/ocaml,gerdstolpmann/ocaml,gerdstolpmann/ocaml