<p><a href="https://en.wikipedia.org/wiki/BEAM_(Erlang_virtual_machine)" rel="nofollow noreferrer">BEAM (Erlang virtual machine)</a> is known for having the ability to run huge amounts of concurrent processes, and very large amounts of messages between those processes. What runtime strategies does it utilise in order to be able to do this? What kind of concurrent main loop and process structure is it using that differentiates it from things like Go's coroutines and (popular implementations of) Rust's async?</p>
<p>(As a bonus: Could these features be implemented natively, without a VM runtime?)</p>

<runtime><runtime-performance><concurrency>
What runtime strategies make BEAM so good at concurrency?
CC BY-SA 4.0
2024-06-25T15:21:26.447Z
One key thing that differentiate BEAM: It implements VM level preemption using a bytecode. After every N bytecode operations, a task is cooperatively suspended by the bytecode engine. Guaranteeing that in an arbitrary ahead of time compiler that allows directly calling into C is a lot harder.
user1937198
CC BY-SA 4.0
2024-06-25T20:18:16.980Z
Another key thing is that BEAM does GC per "process" and not the whole system at a time as most other GCs does.
Jonas
CC BY-SA 4.0
2024-06-25T20:42:09.380Z
If I remember correctly, while Erjang definitely *did* struggle to compete with BEAM, it was not that far off either. So, there is existence proof of an implementation on the Java platform that was that far off.
Jörg W Mittag
<h3>The book</h3>
<p>Let's examine this question using <a href="https://blog.stenmans.org/theBeamBook/#_processes" rel="nofollow noreferrer">The Erlang Runtime System</a> book as base. This book describes both Erlang and BEAM in detail.</p>
<h3>Processes, not threads</h3>
<blockquote>
<p>The concept of lightweight processes is the essence of Erlang and the BEAM; they are what makes BEAM stand out from other virtual machines.</p>
</blockquote>
<p>This may be the principal strategy. Threads don't share memory. This means that each process works almost in isolation of other processes <em>problems</em>. For example, imagine an memory sharing VM running 100,000 threads, where <em>one</em> of them is very &quot;trashy&quot;, that is, this only thread works by filling the entire memory with temporary ciclic objects, to the exhaustion, every second.</p>
<p>In this scenario, a sharing memory VM would, at every second:</p>
<ol>
<li>Waste time to stop all 100,000, progress is slowed here;</li>
<li>Run the GC on entire VM process, no progress is made here;</li>
<li>After unblocking all 100,000 threads, some thread may run until...</li>
<li>The trashy thread runs, exhausting the entire memory again, GOTO 1.</li>
</ol>
<p>Something like <a href="https://github.com/golang/go/issues/17831" rel="nofollow noreferrer">this</a>:
<a href="https://i.sstatic.net/nub5zrNP.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/nub5zrNP.png" alt="Go's issue 17831: a simple tight loop bogs down thousands of routines" /></a></p>
<p>So many CPUs, doing a lot of nothing, a lot of time...</p>
<p>In the BEAM, with each process working in isolation, this would not happen. On the section about <a href="https://blog.stenmans.org/theBeamBook/#_concurrency_parallelism_and_preemptive_multitasking" rel="nofollow noreferrer">Concurrency, Parallelism, and Preemptive Multitasking</a>, there is this example:</p>
<pre><code>[{[{status,garbage_collecting}],&lt;0.62.0&gt;},
 {[{status,garbage_collecting}],&lt;0.66.0&gt;},
 {[{status,runnable}],&lt;0.60.0&gt;},
 {[{status,runnable}],&lt;0.61.0&gt;},
 {[{status,runnable}],&lt;0.63.0&gt;},
 {[{status,runnable}],&lt;0.65.0&gt;},
 {[{status,runnable}],&lt;0.67.0&gt;},
 {[{status,running}],&lt;0.58.0&gt;},
 {[{status,running}],&lt;0.64.0&gt;},
 {[{status,waiting}],&lt;0.0.0&gt;},
 {[{status,waiting}],&lt;0.1.0&gt;},
...
</code></pre>
<p>In other words, some processes are running the garbage collector side by side with a normal running process, making progress in parallel to the garbage collection.</p>
<p>Also note this is a <strong>powerful notion to VM to have.</strong> If the VM can observe some processes are more trashy than others, it then controls the shared memory pool more efficiently by:</p>
<ol>
<li><p>Giving less memory to more trashy processes: this is optimal, as the GC is effective in trashy processes (more memory is released) and by keeping the memory space small (faster GC time);</p>
</li>
<li><p>Giving more memory for less trashy processes: this is optimal, by allowing these processes to make progress instead of burning time in GC cycles that release less memory;</p>
</li>
<li><p><strong>To avoid trashy processes from stealing all VMs memory.</strong></p>
</li>
</ol>
<p>Let me emphasize this a little. The BEAM <strong>never</strong> stops the world (STW). All threads of execution have fully separated memory pages. The garage collector always runs on each process subset, and never runs on the entire VM. This means each GC will run faster, because:</p>
<ol>
<li><p>There is less memory to inspect on each process (in comparison of the entire STW VM);</p>
</li>
<li><p>There is <strong>no</strong> coordination involved.</p>
</li>
</ol>
<p>There are no maps of live registers set per instruction generated, there are no worries about safe or unsafe points of <em>other</em> threads, there are no worries of hardware or software interrupts.</p>
<p>Each process always trips the GC in a known state, and gets no interference from outside. This <strong>massively</strong> simplifies the GC code.</p>
<h3>Message passing without exposed locks</h3>
<blockquote>
<p>Each process has its own memory (a mailbox, a heap and a stack)</p>
</blockquote>
<p>The heap and stack are common notions. The &quot;mailbox&quot; thinguie can be approximated to a buffered, typed <code>stdin</code> of a process, where data is placed in serialized form.</p>
<p>When a process calls another process, the VM internally serializes the call data and places this into target process reach. No memory pointers are shared between processes.</p>
<p>To avoid allocating heap in each remote call, first the VM tries to take a pair wise lock on both processes (to write de plain data directly into the target mailbox), and failing that, allocates a heap object, serializes the call data, and places this separated &quot;mail&quot; into a linked list in the control of target process. Again, no pointer data is shared between processes.</p>
<p>This may sound overkill, but consider an exaggerated example of 99,999 processes trying to talk with one central process. This would mean locks in a sharing memory VM. Worst case scenario, this would mean one process taking a lock, executing the communication <strong>and other things, maybe trying to take other locks (and possibly deadlocking)</strong>, while 99,998 threads wait.</p>
<p>Even in the BEAM, where the central process executes in serial, all 99.999 other processes will make some progress in parallel (the work of serializing the message) and may end fighting for the central's mailbox lock, <strong>without locking</strong> the central process progress. The central process can access the data <strong>without</strong> taking any locks, and will work faster because of that. Also, the central process will not be blocked for &quot;other things&quot; time.</p>
<p>As no lock is exposed to client code, this means that deadlocks are almost impossible in BEAM (but is possible to deadlock progress by memory exhaustion <a href="https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/mchri-tfp-2011.pdf" rel="nofollow noreferrer">and some other patterns</a>).</p>
<h3>There is no locks, there is no mutexes</h3>
<p>Each BEAM process is single threaded. Single threaded code never executes two parts at same time, so <strong>there is no need for locks or mutexes</strong>.</p>
<p>When some code needs to be run as a critical region, you pull this code into a separate process. A BEAM process is already isolated from... everything, and code without a <code>receive</code> will run uninterrupted to completion, so again <strong>there is no need for locks or mutexes</strong>.</p>
<p>It's not so easy to replace multi lock step code with multi agent coordination code, but in principle, code <strong>without</strong> locks will run much faster than code littered with locks and mutexes. And this is good for concurrency <strong>and</strong> parallelism.</p>
<p>And you can always create an <a href="https://github.com/nitrogen/mutagen" rel="nofollow noreferrer">mutex server in pure BEAM</a>.</p>
<h3>Bonus</h3>
<blockquote>
<p>Could these features be implemented natively, without a VM runtime?</p>
</blockquote>
<p><strong>Yes.</strong> This is an example I came up with not long ago. In a terminal Linux console:</p>
<pre><code>cat file.txt | sort | uniq | wc
</code></pre>
<p>These four<sup>*</sup> native processes will run perfectly in machines with any number of CPUs, from 1 to +infinite.</p>
<p>The insight here is that BEAM does not share memory or locks between processes, but <strong>share type information.</strong> All message passing is automatic because the processes use the same code for message serialization and deserialization. With a &quot;common repository&quot; of message formats it would be perfectly possible to recreate almost all of BEAM characteristics in native code that communicates exclusivity by pipes or sockets.</p>
<p><sup><sup>*</sup> Yes, I know about <code>uniq -u</code>.</sup></p>

CC BY-SA 4.0
2024-07-02T22:34:56.457Z
2024-07-14T20:52:24.277Z
2024-07-14T20:52:24.277Z
André L F S Bacci
CC BY-SA 4.0
2024-07-10T15:22:42.780Z
I would note that Concurrent Garbage Collection _does_ exist, and the STW pauses in the JVM are really small by now. Not that heap isolation doesn't have advantages, but the ability to GC in parallel with running the task, compared to having to pause the Beam thread to GC its heap, may have their own advantages. The trade-off seems less clear to me than this answer assumes.
Matthieu M.
CC BY-SA 4.0
2024-07-13T11:48:25.017Z
It sounds a bit like the benefits do not come from smart tricks in the BEAM VM itself, but are consequences of Erlang's design and its "isolated processes with messages" approach. But maybe making this distinction is not relevant.
Eldritch Conundrum
# What runtime strategies make BEAM so good at concurrency?
Posted by: username_0 on 2024-06-24
[BEAM (Erlang virtual machine)](https://en.wikipedia.org/wiki/BEAM_(Erlang_virtual_machine)) is known for having the ability to run huge amounts of concurrent processes, and very large amounts of messages between those processes. What runtime strategies does it utilise in order to be able to do this? What kind of concurrent main loop and process structure is it using that differentiates it from things like Go's coroutines and (popular implementations of) Rust's async?

(As a bonus: Could these features be implemented natively, without a VM runtime?)
**Tags:** runtime runtime-performance concurrency
> **Comments:**
> **username_1**: One key thing that differentiate BEAM: It implements VM level preemption using a bytecode. After every N bytecode operations, a task is cooperatively suspended by the bytecode engine. Guaranteeing that in an arbitrary ahead of time compiler that allows directly calling into C is a lot harder.  
> **username_2**: Another key thing is that BEAM does GC per "process" and not the whole system at a time as most other GCs does.  
> **username_3**: If I remember correctly, while Erjang definitely *did* struggle to compete with BEAM, it was not that far off either. So, there is existence proof of an implementation on the Java platform that was that far off.  
---
## 1 answers
### Answer by username_4 on 2024-07-02. Score: 10 | Accepted answer
The book
Let's examine this question using The Erlang Runtime System book as base. This book describes both Erlang and BEAM in detail.
Processes, not threads

The concept of lightweight processes is the essence of Erlang and the BEAM; they are what makes BEAM stand out from other virtual machines.

This may be the principal strategy. Threads don't share memory. This means that each process works almost in isolation of other processes problems. For example, imagine an memory sharing VM running 100,000 threads, where one of them is very "trashy", that is, this only thread works by filling the entire memory with temporary ciclic objects, to the exhaustion, every second.
In this scenario, a sharing memory VM would, at every second:

Waste time to stop all 100,000, progress is slowed here;
Run the GC on entire VM process, no progress is made here;
After unblocking all 100,000 threads, some thread may run until...
The trashy thread runs, exhausting the entire memory again, GOTO 1.

Something like this:

So many CPUs, doing a lot of nothing, a lot of time...
In the BEAM, with each process working in isolation, this would not happen. On the section about Concurrency, Parallelism, and Preemptive Multitasking, there is this example:
[{[{status,garbage_collecting}],<0.62.0>},
 {[{status,garbage_collecting}],<0.66.0>},
 {[{status,runnable}],<0.60.0>},
 {[{status,runnable}],<0.61.0>},
 {[{status,runnable}],<0.63.0>},
 {[{status,runnable}],<0.65.0>},
 {[{status,runnable}],<0.67.0>},
 {[{status,running}],<0.58.0>},
 {[{status,running}],<0.64.0>},
 {[{status,waiting}],<0.0.0>},
 {[{status,waiting}],<0.1.0>},
...

In other words, some processes are running the garbage collector side by side with a normal running process, making progress in parallel to the garbage collection.
Also note this is a powerful notion to VM to have. If the VM can observe some processes are more trashy than others, it then controls the shared memory pool more efficiently by:

Giving less memory to more trashy processes: this is optimal, as the GC is effective in trashy processes (more memory is released) and by keeping the memory space small (faster GC time);

Giving more memory for less trashy processes: this is optimal, by allowing these processes to make progress instead of burning time in GC cycles that release less memory;

To avoid trashy processes from stealing all VMs memory.


Let me emphasize this a little. The BEAM never stops the world (STW). All threads of execution have fully separated memory pages. The garage collector always runs on each process subset, and never runs on the entire VM. This means each GC will run faster, because:

There is less memory to inspect on each process (in comparison of the entire STW VM);

There is no coordination involved.


There are no maps of live registers set per instruction generated, there are no worries about safe or unsafe points of other threads, there are no worries of hardware or software interrupts.
Each process always trips the GC in a known state, and gets no interference from outside. This massively simplifies the GC code.
Message passing without exposed locks

Each process has its own memory (a mailbox, a heap and a stack)

The heap and stack are common notions. The "mailbox" thinguie can be approximated to a buffered, typed stdin of a process, where data is placed in serialized form.
When a process calls another process, the VM internally serializes the call data and places this into target process reach. No memory pointers are shared between processes.
To avoid allocating heap in each remote call, first the VM tries to take a pair wise lock on both processes (to write de plain data directly into the target mailbox), and failing that, allocates a heap object, serializes the call data, and places this separated "mail" into a linked list in the control of target process. Again, no pointer data is shared between processes.
This may sound overkill, but consider an exaggerated example of 99,999 processes trying to talk with one central process. This would mean locks in a sharing memory VM. Worst case scenario, this would mean one process taking a lock, executing the communication and other things, maybe trying to take other locks (and possibly deadlocking), while 99,998 threads wait.
Even in the BEAM, where the central process executes in serial, all 99.999 other processes will make some progress in parallel (the work of serializing the message) and may end fighting for the central's mailbox lock, without locking the central process progress. The central process can access the data without taking any locks, and will work faster because of that. Also, the central process will not be blocked for "other things" time.
As no lock is exposed to client code, this means that deadlocks are almost impossible in BEAM (but is possible to deadlock progress by memory exhaustion and some other patterns).
There is no locks, there is no mutexes
Each BEAM process is single threaded. Single threaded code never executes two parts at same time, so there is no need for locks or mutexes.
When some code needs to be run as a critical region, you pull this code into a separate process. A BEAM process is already isolated from... everything, and code without a receive will run uninterrupted to completion, so again there is no need for locks or mutexes.
It's not so easy to replace multi lock step code with multi agent coordination code, but in principle, code without locks will run much faster than code littered with locks and mutexes. And this is good for concurrency and parallelism.
And you can always create an mutex server in pure BEAM.
Bonus

Could these features be implemented natively, without a VM runtime?

Yes. This is an example I came up with not long ago. In a terminal Linux console:
cat file.txt | sort | uniq | wc

These four* native processes will run perfectly in machines with any number of CPUs, from 1 to +infinite.
The insight here is that BEAM does not share memory or locks between processes, but share type information. All message passing is automatic because the processes use the same code for message serialization and deserialization. With a "common repository" of message formats it would be perfectly possible to recreate almost all of BEAM characteristics in native code that communicates exclusivity by pipes or sockets.
* Yes, I know about uniq -u.
> **Comments:**
> **username_5** (2024-07-10): I would note that Concurrent Garbage Collection _does_ exist, and the STW pauses in the JVM are really small by now. Not that heap isolation doesn't have advantages, but the ability to GC in parallel with running the task, compared to having to pause the Beam thread to GC its heap, may have their own advantages. The trade-off seems less clear to me than this answer assumes.  
> **username_6** (2024-07-13): It sounds a bit like the benefits do not come from smart tricks in the BEAM VM itself, but are consequences of Erlang's design and its "isolated processes with messages" approach. But maybe making this distinction is not relevant.  
---
