<h2>Overview</h2>
<p>Ensuring memory safety is a core facet of modern programming language design. Memory safety can be guaranteed in many different ways.</p>
<p>For the purposes of this question, I am defining memory safety as the prevention of the following bugs: Note that a language may provide an 'unsafe' workaround, thereby forsaking the memory safety guarantor.</p>
<ul>
<li><strong>Use-after-free (UAF) and double-free</strong>: Usage of a memory region after its release.</li>
<li><strong>Out of bounds read</strong>: Reading from memory outside of a bounded region.</li>
<li><strong>Out of bounds write</strong>: Writing to memory outside of a bounded region.</li>
<li><strong>Null pointer dereferencing</strong>: Accessing a pointer that is semantically defined not to be valid.</li>
<li><strong>Runtime type confusion</strong>: Interpreting a pointer to some memory as the wrong type.</li>
</ul>
<p><em>This list may be incomplete.</em>
Source: <a href="https://sternumiot.com/iot-blog/memory-safety-5-common-memory-bugs-and-how-to-secure-your-system/" rel="nofollow noreferrer">https://sternumiot.com/iot-blog/memory-safety-5-common-memory-bugs-and-how-to-secure-your-system/</a></p>
<p>Any compiled or interpreted language may prevent the following without <strong>complex fundamental changes</strong>.</p>
<ul>
<li><strong>Out-of-bounds read</strong>: Compiler inserts array bounds checking on every access.</li>
<li><strong>Out-of-bounds write</strong>: Compiler inserts array bounds checking on every access.</li>
<li><strong>Null pointer dereferencing</strong>: Compiler compares pointer to null on every access.</li>
<li><strong>Overflow and underflow</strong>: The compiler checks the intermediate of every expression to ensure it is in range.</li>
</ul>
<p>Modern programming languages like Go and Rust tend to have more creative solutions to preventing the following classes:</p>
<ul>
<li><strong>Use-after-free (UAF)</strong></li>
<li><strong>Runtime type confusion</strong></li>
</ul>
<h3>Question 1: What are the primary semantic classes of languages that prevent Use-after-free (UAF) and runtime-type confusion errors in a manner that guarantees memory safety?</h3>
<p>Here are a few broad classes of programming language semantics that appear to prevent Use-after-frees:</p>
<ul>
<li>Purely functional languages with no global state.</li>
<li>Garbage-collected languages that handle the release of memory and that do not provide a 'safe' way to free that memory manually.</li>
<li>Reference-counted languages that handle memory release and that do not provide a 'safe' way to free that memory manually or adjust the reference count.</li>
<li>Borrow checking.</li>
</ul>
<h3>Are there any others?</h3>

<memory-management><garbage-collection><memory-safety><pointers><references>
Analysis of methods to ensure memory safety
CC BY-SA 4.0
2024-10-18T20:57:58.730Z
Languages that just leak memory (reasonable for short-running programs), languages that store everything on the stack, and interpreted languages that inherit their host's memory management.
tarzh
CC BY-SA 4.0
2024-10-19T02:32:05.700Z
https://verdagon.dev/grimoire/grimoire
apropos
CC BY-SA 4.0
2024-10-19T05:34:13.133Z
I am looking for ways ways to do this in a statically typed / compiled language that makes all raw pointers unsafe. It provides arrays (fixed size) and dynamic memory facilities. I'm on the fence of whether to use a ref-counted or garbage collected system. I'm interested in exploring a hybrid approach whereby the compiler will perform Rust-like borrow-checking and if the lifetimes are 'trivial' it will just insert the manual 'free()'s', otherwise it will use ref-counted or GC for non-trivial lifetimes. This way Rusty programs will have no overhead and the memory safety will be transparent.
Wesley Jones
CC BY-SA 4.0
2024-10-19T05:39:09.480Z
So in a case where the rust compiler would usually emit a lifetime error, my compiler would implicitly utilize an atomic reference counted wrapper around the offending object. This approach may even permit the trans-compilation of valid rust programs into my language directly without hurting performance. This way I could reuse the Rust Libraries and not have to rewrite the world.
Wesley Jones
CC BY-SA 4.0
2024-10-19T06:57:38.743Z
@WesleyJones I think this might be a good approach for memory safety. Just that I think mixing Rust-style lifetime management with ref counting might be hard... (I'm working on a new programming language as well FYI and find this to be the hardest problem.) I recommend looking at the memory management of Vale. For the question above, I wonder why you consider underflow / overflow a _memory_ safety issue?
Thomas Mueller
CC BY-SA 4.0
2024-10-19T11:14:52.597Z
It is _quite_ strange to classify overflow & underflow as a memory safety issue. For example, Rust is considered memory safe, yet allows overflow & underflow (in Release, by default).
Matthieu M.
CC BY-SA 4.0
2024-10-19T12:45:08.083Z
Why does this have separate sections "Abstract" and "Question 1" but nothing else? The headings seem rather non-indicative and misleading: the abstract doesn’t appear to be an, well, abstract of the question and numbering just a single question seems out of place.
MisterMiyagi
CC BY-SA 4.0
2024-10-21T15:46:31.127Z
In general "memory safety" requires that the storage associated with some variable always holds a bit pattern representing a valid member of that variable's type. Overflow and underflow don't cause that problem for standard integer types, but in principle they could for a range-restricted integer type.
kaya3
<p>In the question, you note that runtime checks can be used to prevent out-of-bounds reads and writes. It is worth noting that out-of-bounds access can only be checked for at runtime if the bounds are known at runtime; this fundamentally doesn't work with C-style arrays, which (unlike arrays in e.g. Java) are not length-prefixed.</p>
<p>That is, for certain kinds of runtime checking, there must exist at runtime some data associated with the property to be checked. Likewise, if type information is available at runtime, then runtime type confusion can be prevented via runtime checks.</p>
<p>It is possible to use the same approach to detect use-after-free (UAF) at runtime, too. For this, each allocated region of memory can be prefixed with a &quot;generation number&quot;, which is the number of times that address has been used for a new allocation. Then, references are &quot;fat pointers&quot; holding an address and an expected generation number. When dereferencing, the expected generation number from the fat pointer is compared with the generation number stored at that address, and the dereference fails if they are not equal. So long as every (address, generation number) pair is used for at most one allocation for the lifetime of the program, this guarantees to prevent UAF.</p>
<p>The use of generation numbers to detect UAF at runtime is <a href="https://langdev.stackexchange.com/a/1352/89">how Vale makes weak references work</a>. Weak references are a similar problem to preventing UAF, because in both cases you have an address and you want to know whether what's at that address has the same &quot;identity&quot; as when the address was taken. So it's relevant to look at how weak references are implemented in various langauges, since those have similar memory-safety issues.</p>

CC BY-SA 4.0
2024-10-18T20:11:53.417Z
2024-10-18T20:11:53.417Z
kaya3
<p>Use-after-free can be prevented via generational references (see Kaya3's response), or via a more traditional weak reference approaches, where there is a hash map of all valid weak references, and access is over this hash map. There are multiple ways.</p>
<p>Type confusion can be prevented by the compiler by either disallowing unsafe casts, and checks at runtime (for union, the compiler would need to use tagged unions, and check the tag on every access).</p>
<p>This all comes with a cost; partial incompatibility with existing code and existing compilers (if you eg. add it to a C compiler). Overflow and underflow checking (I wouldn't consider that a memory safety issue btw), if done at runtime, also has a slightly high cost. I guess that's the reason why there is a limited number of C compilers that uses some of the techniques: it would slow execution down, and cause incompatibility. If you are designing a new language, this applies as well: if you want to employ all these checks at runtime, then execution might be slow, and I guess people might shy away from this. If you want to do them at compile time, then the language might be hard to use. That, I think, is the challenging problem when designing a new language.</p>

CC BY-SA 4.0
2024-10-19T07:11:39.813Z
2024-10-19T07:11:39.813Z
Thomas Mueller
# Analysis of methods to ensure memory safety
Posted by: username_0 on 2024-10-18
## Overview

Ensuring memory safety is a core facet of modern programming language design. Memory safety can be guaranteed in many different ways.

For the purposes of this question, I am defining memory safety as the prevention of the following bugs: Note that a language may provide an 'unsafe' workaround, thereby forsaking the memory safety guarantor.

* **Use-after-free (UAF) and double-free**: Usage of a memory region after its release.
* **Out of bounds read**: Reading from memory outside of a bounded region.
* **Out of bounds write**: Writing to memory outside of a bounded region.
* **Null pointer dereferencing**: Accessing a pointer that is semantically defined not to be valid.
* **Runtime type confusion**: Interpreting a pointer to some memory as the wrong type.

*This list may be incomplete.*
Source: <https://sternumiot.com/iot-blog/memory-safety-5-common-memory-bugs-and-how-to-secure-your-system/>

Any compiled or interpreted language may prevent the following without **complex fundamental changes**.

* **Out-of-bounds read**: Compiler inserts array bounds checking on every access.
* **Out-of-bounds write**: Compiler inserts array bounds checking on every access.
* **Null pointer dereferencing**: Compiler compares pointer to null on every access.
* **Overflow and underflow**: The compiler checks the intermediate of every expression to ensure it is in range.

Modern programming languages like Go and Rust tend to have more creative solutions to preventing the following classes:

* **Use-after-free (UAF)**
* **Runtime type confusion**

### Question 1: What are the primary semantic classes of languages that prevent Use-after-free (UAF) and runtime-type confusion errors in a manner that guarantees memory safety?

Here are a few broad classes of programming language semantics that appear to prevent Use-after-frees:

* Purely functional languages with no global state.
* Garbage-collected languages that handle the release of memory and that do not provide a 'safe' way to free that memory manually.
* Reference-counted languages that handle memory release and that do not provide a 'safe' way to free that memory manually or adjust the reference count.
* Borrow checking.

### Are there any others?
**Tags:** memory-management garbage-collection memory-safety pointers references
> **Comments:**
> **username_1**: Languages that just leak memory (reasonable for short-running programs), languages that store everything on the stack, and interpreted languages that inherit their host's memory management.  
> **username_2**: https://verdagon.dev/grimoire/grimoire  
> **username_0**: I am looking for ways ways to do this in a statically typed / compiled language that makes all raw pointers unsafe. It provides arrays (fixed size) and dynamic memory facilities. I'm on the fence of whether to use a ref-counted or garbage collected system. I'm interested in exploring a hybrid approach whereby the compiler will perform Rust-like borrow-checking and if the lifetimes are 'trivial' it will just insert the manual 'free()'s', otherwise it will use ref-counted or GC for non-trivial lifetimes. This way Rusty programs will have no overhead and the memory safety will be transparent.  
> **username_0**: So in a case where the rust compiler would usually emit a lifetime error, my compiler would implicitly utilize an atomic reference counted wrapper around the offending object. This approach may even permit the trans-compilation of valid rust programs into my language directly without hurting performance. This way I could reuse the Rust Libraries and not have to rewrite the world.  
> **username_3**: @WesleyJones I think this might be a good approach for memory safety. Just that I think mixing Rust-style lifetime management with ref counting might be hard... (I'm working on a new programming language as well FYI and find this to be the hardest problem.) I recommend looking at the memory management of Vale. For the question above, I wonder why you consider underflow / overflow a _memory_ safety issue?  
> **username_4**: It is _quite_ strange to classify overflow & underflow as a memory safety issue. For example, Rust is considered memory safe, yet allows overflow & underflow (in Release, by default).  
> **username_5**: Why does this have separate sections "Abstract" and "Question 1" but nothing else? The headings seem rather non-indicative and misleading: the abstract doesn’t appear to be an, well, abstract of the question and numbering just a single question seems out of place.  
> **username_6**: In general "memory safety" requires that the storage associated with some variable always holds a bit pattern representing a valid member of that variable's type. Overflow and underflow don't cause that problem for standard integer types, but in principle they could for a range-restricted integer type.  
---
## 2 answers
### Answer by username_6 on 2024-10-18. Score: 7
In the question, you note that runtime checks can be used to prevent out-of-bounds reads and writes. It is worth noting that out-of-bounds access can only be checked for at runtime if the bounds are known at runtime; this fundamentally doesn't work with C-style arrays, which (unlike arrays in e.g. Java) are not length-prefixed.

That is, for certain kinds of runtime checking, there must exist at runtime some data associated with the property to be checked. Likewise, if type information is available at runtime, then runtime type confusion can be prevented via runtime checks.

It is possible to use the same approach to detect use-after-free (UAF) at runtime, too. For this, each allocated region of memory can be prefixed with a "generation number", which is the number of times that address has been used for a new allocation. Then, references are "fat pointers" holding an address and an expected generation number. When dereferencing, the expected generation number from the fat pointer is compared with the generation number stored at that address, and the dereference fails if they are not equal. So long as every (address, generation number) pair is used for at most one allocation for the lifetime of the program, this guarantees to prevent UAF.

The use of generation numbers to detect UAF at runtime is [how Vale makes weak references work](https://langdev.stackexchange.com/a/1352/89). Weak references are a similar problem to preventing UAF, because in both cases you have an address and you want to know whether what's at that address has the same "identity" as when the address was taken. So it's relevant to look at how weak references are implemented in various langauges, since those have similar memory-safety issues.
---
### Answer by username_3 on 2024-10-19. Score: 3
Use-after-free can be prevented via generational references (see Kaya3's response), or via a more traditional weak reference approaches, where there is a hash map of all valid weak references, and access is over this hash map. There are multiple ways.

Type confusion can be prevented by the compiler by either disallowing unsafe casts, and checks at runtime (for union, the compiler would need to use tagged unions, and check the tag on every access).

This all comes with a cost; partial incompatibility with existing code and existing compilers (if you eg. add it to a C compiler). Overflow and underflow checking (I wouldn't consider that a memory safety issue btw), if done at runtime, also has a slightly high cost. I guess that's the reason why there is a limited number of C compilers that uses some of the techniques: it would slow execution down, and cause incompatibility. If you are designing a new language, this applies as well: if you want to employ all these checks at runtime, then execution might be slow, and I guess people might shy away from this. If you want to do them at compile time, then the language might be hard to use. That, I think, is the challenging problem when designing a new language.
---
