<p>I am making an interpreter based on the lox from the book <a href="https://craftinginterpreters.com/" rel="nofollow noreferrer">Crafting Interpretes</a>. I have modified the language implementation heavily but the core still remains the same.</p>
<p>Currently I am adding standard library to the implementation. This requires storing C structs in a class or instance. For example I am using <code>pcre2</code> for regex, so I wanted to provide a compile function which stores <code>pcre2_code *</code> in a class (the <code>pcre2_code</code> struct is not defined in headers and only forward declared). What I have thought of is creating a new object like</p>
<pre class="lang-c prettyprint-override"><code>typedef struct {
    Obj obj;
    void *ptr;
    size_t size;
} ObjNativeStruct;
</code></pre>
<p>Then I simply convert the <code>void *</code> to the required struct and optionally check the size using <code>sizeof(target_struct) == obj-&gt;size</code>.</p>
<p>Is this implementation correct and secure (Secure as in the sense of converting the <code>void *</code> to a struct with larger size and potentially accessing unowned memory)? Is there any alternative to my current implementation (I also thought about ffi, but at the moment they seem too complex).</p>
<p>Note: <a href="https://github.com/blood-rogue/lox" rel="nofollow noreferrer">Link to the repo</a></p>

<compilers><c><interpreters><pointers>
How to save C struct pointers in interpreted language?
CC BY-SA 4.0
2024-03-16T21:14:32.977Z
It is unclear to me what you mean by "adding standard library requires storing C structs in a class or instance"?
Eldritch Conundrum
CC BY-SA 4.0
2024-03-17T01:47:42.917Z
Security is not "yes it is secure" or "no it is not secure".  Secure **against what threat**? What resource is being attacked, who owns it, who is threatening it? These are just some of the questions that must be answered when you write your *threat model*, so start by doing some research on threat models.
Eric Lippert
CC BY-SA 4.0
2024-03-17T09:28:31.400Z
If I understand your edits: your standard library will not be written in Lox, so you're implementing a Foreign Function Interface for C, and you're wondering how to handle the memory allocated from C?
Eldritch Conundrum
CC BY-SA 4.0
2024-03-17T10:42:06.117Z
@EldritchConundrum Well you are kinda correct. While I am writting the std in C, it is not a full fledged ffi, i.e. others cannot utilize it to make there own modules. The modules are predefined and built into the binary.
Timsib Adnap
<p>Take a look at how another one of Nystrom's languages does it: In Wren, there's a struct <code>ObjForeign</code> (<a href="https://github.com/wren-lang/wren/blob/main/src%2Fvm%2Fwren_value.h#L418-L421" rel="nofollow noreferrer">source</a>) containing the base <code>Obj</code> and an opaque <code>uint8_t[FLEXIBLE_ARRAY] data</code> (see <a href="https://github.com/wren-lang/wren/blob/main/src%2Fvm%2Fwren_common.h#L139-L148" rel="nofollow noreferrer">here</a> for how <code>FLEXIBLE_ARRAY</code> works).</p>
<p>If you read through Wren's article on <a href="https://wren.io/embedding/storing-c-data.html" rel="nofollow noreferrer">Storing C Data</a>, you'll find that it doesn't even store the size – the user has to make sure the types match up (just like when casting any other <code>void*</code> in C). There must be an according class object though (defined in Wren or through callbacks).</p>

CC BY-SA 4.0
2024-03-17T09:27:52.387Z
2024-03-17T09:27:52.387Z
linux_user36
CC BY-SA 4.0
2024-03-17T10:39:31.250Z
This is new for me. BTW The structs I am using are incomplete i.e. are not complete in the headers. So how do I allocate memory to store the pointer?
Timsib Adnap
CC BY-SA 4.0
2024-03-17T11:16:08.267Z
@TimsibAdnap Simply use `malloc/calloc` with the base struct size plus N and cast the result to your `ObjNativeStruct`. It's called the "struct hack" and is technically undefined behavior, though it works on basically any compiler (see [https://stackoverflow.com/q/3711233/12344762]). Because the data is inlined, both allocation and deallocation must be handled by the VM, and as far as I can tell, the affected objects are excluded from GC.
linux_user36
CC BY-SA 4.0
2024-03-17T11:26:49.293Z
The issue lies in the fact that I do not control the allocation of the struct, and also I do not know the size of the struct as it is incomplete in the headers and declared else where. I am using an external library (`pcre2` for reference) which allocates the struct and return to me a pointer.
Timsib Adnap
CC BY-SA 4.0
2024-03-17T11:32:03.680Z
@TimsibAdnap In that case I'd recommend your current concept, though you might get rid of the `size` field, as type-checking should be done via the object system anyway (and two objects randomly matching in size isn't very secure for that purpose).
linux_user36
CC BY-SA 4.0
2024-03-17T11:46:56.823Z
Ok and thanks for the answer as I will be using it afterwards.
Timsib Adnap
# How to save C struct pointers in interpreted language?
Posted by: username_0 on 2024-03-16
I am making an interpreter based on the lox from the book Crafting Interpretes. I have modified the language implementation heavily but the core still remains the same.
Currently I am adding standard library to the implementation. This requires storing C structs in a class or instance. For example I am using pcre2 for regex, so I wanted to provide a compile function which stores pcre2_code * in a class (the pcre2_code struct is not defined in headers and only forward declared). What I have thought of is creating a new object like
typedef struct {
    Obj obj;
    void *ptr;
    size_t size;
} ObjNativeStruct;

Then I simply convert the void * to the required struct and optionally check the size using sizeof(target_struct) == obj->size.
Is this implementation correct and secure (Secure as in the sense of converting the void * to a struct with larger size and potentially accessing unowned memory)? Is there any alternative to my current implementation (I also thought about ffi, but at the moment they seem too complex).
Note: Link to the repo
**Tags:** compilers c interpreters pointers
> **Comments:**
> **username_1**: It is unclear to me what you mean by "adding standard library requires storing C structs in a class or instance"?  
> **username_2**: Security is not "yes it is secure" or "no it is not secure".  Secure **against what threat**? What resource is being attacked, who owns it, who is threatening it? These are just some of the questions that must be answered when you write your *threat model*, so start by doing some research on threat models.  
> **username_1**: If I understand your edits: your standard library will not be written in Lox, so you're implementing a Foreign Function Interface for C, and you're wondering how to handle the memory allocated from C?  
> **username_0**: @EldritchConundrum Well you are kinda correct. While I am writting the std in C, it is not a full fledged ffi, i.e. others cannot utilize it to make there own modules. The modules are predefined and built into the binary.  
---
## 1 answers
### Answer by username_3 on 2024-03-17. Score: 3 | Accepted answer
Take a look at how another one of Nystrom's languages does it: In Wren, there's a struct `ObjForeign` ([source](https://github.com/wren-lang/wren/blob/main/src%2Fvm%2Fwren_value.h#L418-L421)) containing the base `Obj` and an opaque `uint8_t[FLEXIBLE_ARRAY] data` (see [here](https://github.com/wren-lang/wren/blob/main/src%2Fvm%2Fwren_common.h#L139-L148) for how `FLEXIBLE_ARRAY` works).

If you read through Wren's article on [Storing C Data](https://wren.io/embedding/storing-c-data.html), you'll find that it doesn't even store the size – the user has to make sure the types match up (just like when casting any other `void*` in C). There must be an according class object though (defined in Wren or through callbacks).
> **Comments:**
> **username_0** (2024-03-17): This is new for me. BTW The structs I am using are incomplete i.e. are not complete in the headers. So how do I allocate memory to store the pointer?  
> **username_3** (2024-03-17): @TimsibAdnap Simply use `malloc/calloc` with the base struct size plus N and cast the result to your `ObjNativeStruct`. It's called the "struct hack" and is technically undefined behavior, though it works on basically any compiler (see [https://stackoverflow.com/q/3711233/12344762]). Because the data is inlined, both allocation and deallocation must be handled by the VM, and as far as I can tell, the affected objects are excluded from GC.  
> **username_0** (2024-03-17): The issue lies in the fact that I do not control the allocation of the struct, and also I do not know the size of the struct as it is incomplete in the headers and declared else where. I am using an external library (`pcre2` for reference) which allocates the struct and return to me a pointer.  
> **username_3** (2024-03-17): @TimsibAdnap In that case I'd recommend your current concept, though you might get rid of the `size` field, as type-checking should be done via the object system anyway (and two objects randomly matching in size isn't very secure for that purpose).  
> **username_0** (2024-03-17): Ok and thanks for the answer as I will be using it afterwards.  
---
