<p>Do you think it is possible to have a language that uses garbage collection (GC) by default, but allows you take more control with manual memory management like C++ or Rust, in areas of the software where you care more about performance?</p>
<p>Are there any examples of languages that do this, in academia or the &quot;real world&quot;? Are there particular key problems that need to be overcome?</p>
<p>Maybe the allocation strategy could be a generic type parameter.</p>

<garbage-collection><memory-management>
Possible to mix garbage collection and manual memory management?
CC BY-SA 4.0
2023-09-24T19:32:34.637Z
“Are there any examples of languages that do this”: Nim.
xigoi
CC BY-SA 4.0
2023-09-24T19:43:40.177Z
Another example: early Rust.
Martin Berger
CC BY-SA 4.0
2023-09-24T20:33:34.453Z
C# also does that.
Eldritch Conundrum
CC BY-SA 4.0
2023-09-25T00:08:29.420Z
Expanding on @MartinBerger's comment: early Rust used to have a garbage collector for the things it couldn't verify statically.
Seggan
CC BY-SA 4.0
2023-09-25T14:52:49.830Z
Conservative GC has been bolted on to languages with manual memory management.
Barmar
CC BY-SA 4.0
2023-09-25T20:51:35.930Z
Why are you Asking here whether there languages that do this, in academia or the 'real world'? Why are you not Posting your own research?
Robbie Goodwin
CC BY-SA 4.0
2023-09-26T07:15:51.483Z
There is a new language that is under development, that allows you to specify the GC mechanism as a compile-time option :  [Vlang](https://vlang.io/), there is also a third option which also emits the necessary deallocation code at the compile-time.
Eldar
CC BY-SA 4.0
2023-09-26T11:18:30.767Z
Swift, which uses Reference Counting GC, is another example of a language that also supports dealing with [manual memory management](https://developer.apple.com/documentation/swift/manual-memory-management), mostly for interfacing with C APIs.
DarkDust
CC BY-SA 4.0
2023-09-26T20:39:53.507Z
Just to be clear, what about Python makes it not match your criteria? It uses gc by default, but if you don't allow any references to your object to "escape", then CPython guarantees that it will use reference-counting, and that (if there are no reference loops) your object *will* be destroyed on exit from the scope from which it doesn't escape. It doesn't have a "delete" instruction that can destroy an object despite it still being referenced, but it does have *some* built-in control of object lifetime. How much more control does a language need to provide, in order to qualify?
Steve Jessop
CC BY-SA 4.0
2023-09-26T20:46:14.697Z
I mean, there's also a ton of other reasons not to use Python for the code where you absolutely positively care most about performance. But this isn't a question about the GIL or about real-time guarantees, it's just about avoiding gc. You could say reference-counting is still GC, but then any language that uses reference-counting by default, falls back to manual, and has no way to even clean up reference loops, would qualify. ISTR Perl maybe in that state at some early stage, before mark-sweep was implemented.
Steve Jessop
CC BY-SA 4.0
2023-09-26T23:53:08.877Z
@SteveJessop To your question of why Python doesn't work for me... It might, but it's not ideal. I work on apps with pieces that need high performance, mostly around graphics, computation geometry, math. I haven't found anything that beats C or C++ here, for me. But to write a whole app in it is painful. I also have a vision of building software with an IDE that is more like ...
Rob N
CC BY-SA 4.0
2023-09-26T23:53:30.330Z
... (cont) a notebook environment - interactive, literate programming (Like Matlab, Jupyter) so you intermix code,  documentation, example output. I might have to use Python, but I also really like advanced type systems and trying to prove code works, and on this Python went the other direction.
Rob N
CC BY-SA 4.0
2023-09-27T00:11:10.993Z
There is the "I like notebooks" guy, Jeremy Howard, who does everything in notebooks and nbdev, including developing, documenting and building entirely non-graphical libraries for use by people whether they're in Jupyter or not. I still can't tell whether I think he's right or just ignoring a heap of problems, and I'm not brave enough to find out for myself. FFI is totally the dominant paradigm for performance in the Python ecosystem, though. Stuff like numpy gets approx 0% of its performance gain from avoiding GC, and approx 100% from avoiding allocations and Python bytecodes entirely.
Steve Jessop
CC BY-SA 4.0
2023-09-27T02:28:20.197Z
Thanks. I may end up using that. I've been messing with creating my own language that is interactive, but with types, a JIT, Swift-like syntax. It may just be too crazy hard to do. So for my sanity I may have to drop it. // About the perf gains of memory control, I doubt it's 0%, but Python is so slow the other issues dominate. For some things I found C++ beat Swift. I'm not sure why, but one of the only differences there is more control over memory allocation, and Swift using a lot of reference counting.
Rob N
CC BY-SA 4.0
2023-09-27T17:00:06.080Z
Is it _possible?_ Of course! The trivial way to do it is to have two completely separate heaps: One with GC and one without.
Solomon Slow
CC BY-SA 4.0
2023-09-27T20:10:43.543Z
„one of the only differences there is more control over memory allocation“ – … and the fact that the Swift compiler has been optimized for 13 years by one company whereas C++ compilers have been optimized for 44 years by dozens of companies … including Apple. True, Swift also benefits from all the work poured into LLVM, but then again, so do Rust, Kotlin, Fortran, Ada, D, Delphi, Go, C, and of course C++
Jörg W Mittag
<p>Certainly it is possible, but do people actually want it?</p>
<p>Well, of course we all want &quot;the best of both worlds&quot;. But by and large, each world is big enough by itself to satisfy almost everyone who lives there: languages with manual memory management tend to have standard library or third-party library features for some levels of automatic memory management (e.g. <a href="https://en.wikipedia.org/wiki/Reference_counting" rel="noreferrer">reference counting</a> and <a href="https://en.wikipedia.org/wiki/Region-based_memory_management" rel="noreferrer">arena allocation</a> cover most use-cases); and the performance cost of having a GC for every object is clearly acceptable to many programmers, and often negligible in practice.</p>
<blockquote>
<p>Are there particular key problems that need to be overcome?</p>
</blockquote>
<p>The problems will occur when you try to mix the two. If a program uses exclusively managed objects, or exclusively unmanaged memory allocations, then there is no problem, but in that case there would also be no reason to use a language which supports both ─ you might as well just have two different languages.</p>
<p>It's worth noting that many of these problems also apply to <a href="https://en.wikipedia.org/wiki/Foreign_function_interface" rel="noreferrer">FFI</a>, for example when writing C extensions for Python. This isn't exactly mixing managed and unmanaged memory in the same language, but the C programmer who writes the extension does have to account for Python's garbage collection. The upside is that Python programmers can't accidentally cause problems with unmanaged memory.</p>
<p>So if you want the benefits of a GC'd language but you also want to offer careful programmers the performance benefits of having some manually managed allocations, consider FFI as an alternative language design feature.</p>
<h2>Managed objects holding references to unmanaged memory</h2>
<p>A garbage-collected object which holds a reference to an unmanaged allocation, will have to manually deallocate the unmanaged memory when the managed object is collected. You can do this with <a href="https://en.wikipedia.org/wiki/Finalizer" rel="noreferrer">finalisers</a>, but there are <a href="https://en.wikipedia.org/wiki/Finalizer#Problems" rel="noreferrer">many problems associated with finalisers</a>:</p>
<ul>
<li>They're called non-deterministically in an unreliable order, so you can't guarantee anything about the program state (e.g. object invariants) when they're invoked;</li>
<li>They are invoked by the GC but can change the live-status of other objects, complicating the GC implementation;</li>
<li>Programmers frequently misuse them for non-memory resources;</li>
<li>There's no natural place in the program to catch or handle exceptions or other runtime errors raised in finalisers;</li>
<li>If the GC is multithreaded then finalisers need to be written to take care of this, even if the program itself is single-threaded; deadlocks in finalisers are also possible.</li>
</ul>
<p>These are all reasons that newer languages tend to avoid having finalisers, and older languages are deprecating them (e.g. <a href="https://stackoverflow.com/q/56139760/12299000">Java</a>). But without finalisers, or something like them, GC-managed objects can't own unmanaged memory without leaking it when they're collected.</p>
<p>An alternative to finalisers which solves some problems is to have the GC enqueue notifications about collected objects (i.e. <a href="https://en.wikipedia.org/wiki/Phantom_reference" rel="noreferrer">phantom references</a>) into a user-controlled queue, so that the user's program can choose when to traverse that queue and in which thread; this should be safer overall, but it shifts more work onto the user, and that comes with new potential sources of bugs.</p>
<h2>Unmanaged memory holding references to managed objects</h2>
<p>A manually-managed allocation which holds a reference to a GC-managed object will have to manually inform the GC about it. The GC needs to know about all roots (definitely-live references to managed objects); it can know automatically about roots held by the call stack, but not those held in unmanaged memory. Then, when the unmanaged allocation is deallocated, the programmer needs to again manually inform the GC that the reference has been dropped.</p>
<p>This basically means the programmer must manually write code which would otherwise be generated automatically by a tracing GC.</p>
<h2>Raw memory access of managed objects</h2>
<p>If the language allows reading/writing to memory using raw pointers, then the language must ensure that either</p>
<ul>
<li>Nobody can acquire a pointer to a managed object;</li>
<li>Managed objects have reliable, specified memory layouts (in particular, the object header used by the GC must be a fixed or reliably computed size); or</li>
<li>Otherwise, if programmers can directly read/write to managed allocations, that they know not to do this (although realistically, they will do it anyway).</li>
</ul>
<p>The issue is that writing to an object header will generally lead to <a href="https://en.wikipedia.org/wiki/Undefined_behaviour" rel="noreferrer">undefined behaviour</a> in the sense that anything could happen. The GC is a black box, so a mutated object header could have arbitrary meaning to the GC; a mutated header might violate some invariants, or cause a finaliser to be invoked at the wrong time.</p>

CC BY-SA 4.0
2023-09-24T17:21:15.400Z
2023-09-25T17:58:07.573Z
2023-09-25T17:58:07.573Z
kaya3
CC BY-SA 4.0
2023-09-24T18:08:42.020Z
Very useful answer, thanks. Though I'm not sure I agree that there is "no reason" to use such a language. I think I would like it. ... Good point that a one-language solution would be a lot like an FFI. I've been tinkering with creating my own language. Python *almost* works for me, because I want interactive/notebook development and I can work with their two-language/FFI solution to performance. My problem with Python is the lack of sophisticated static types and analysis.
Rob N
CC BY-SA 4.0
2023-09-24T18:29:14.037Z
@RobN The "no reason" part is if you only ever use one or the other ─ that is, there are problems if you do use both features together, those problems don't occur if you don't mix them but there's no point having a language that lets you mix them if you always avoid doing so.
kaya3
CC BY-SA 4.0
2023-09-25T01:50:53.173Z
There are other options besides finalizers. Things like [PhantomReference](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ref/PhantomReference.html) and reference queues, for example. They don't avoid *all* the problems of finalizers, but they avoid a lot - for example, a PhantomReference can't resurrect objects.
user2357112
CC BY-SA 4.0
2023-09-25T06:54:58.773Z
@RobN: "My problem with Python is the lack of sophisticated static types and analysis." Have you looked at [mypy](https://mypy-lang.org/)?
Oddthinking
CC BY-SA 4.0
2023-09-25T15:34:30.410Z
@user2357112: [*Ephemerons*](https://wikipedia.org/wiki/Ephemeron) sound weakly related (pun intended) as well.
Jörg W Mittag
CC BY-SA 4.0
2023-09-25T21:05:05.757Z
Instead of referring to "unmanaged memory" I think it is more useful to think of resources in terms of *responsibilities* that are acquired, and may only be released by fulfilling them or handing them off.  Code that opens a file acquires a responsibility to close it; code that acquires a lock also acquires a responsibility to release it, etc.  An advantage of a hybrid approach where code manually carries out responsibilities is that the GC can ensure that a reference to an object associated with e.g. a file, will always be a valid reference to something, even if it's an object that's...
supercat
CC BY-SA 4.0
2023-09-25T21:07:46.497Z
...unable to do anything except that it's a permanently useless object.  Also, GC frameworks may allow objects to be pinned, with a proviso that code must ensure that no outside copies of the address will ever be used after an object is unpinned.
supercat
CC BY-SA 4.0
2023-09-26T00:11:55.523Z
I strongly agree with supercat's "responsibilities" framework. When you allocate an unmanaged object, you assume responsibility for its disposal. Some of the objections discussed in your answer represent cases where the programmer is not living up to this responsibility. (Most clearly "unmanaged object owned by a managed object.") You shouldn't (I claim) lay blame for this on the _mixture_ of modes; it's already a consequence of using unmanaged memory at all.
Glenn Willen
CC BY-SA 4.0
2023-09-26T17:26:54.973Z
@GlennWillen The "responsibilities" model does not contradict anything written in my answer; indeed, an object has a responsibility to dispose of any resources it owns, including unmanaged memory, so therefore you need something like a finaliser if a managed object owns an unmanaged allocation. I'm not saying it's a problem that there are responsibilities at all; I'm saying there is a problem when a GC-managed object represents such a responsibility. Therefore this problem *is* specifically associated with mixing managed and unmanaged memory. The same issue occurs when a GC-managed object ...
kaya3
CC BY-SA 4.0
2023-09-26T17:31:00.583Z
... some other resource such as a file handle or network socket; the OOP model of representing things with GC-managed objects is not a good fit for managing responsibilities. Again, it is not fundamentally a problem to *have* resources which create a responsibility to dispose of them, the problem is when you mix that with a paradigm that is not well-suited for disposing of such resources deterministically at definite times. Yes, this means the programmer is not living up to the responsibility, but that particularly happens when the language gives the wrong tools for the programmer to do so.
kaya3
CC BY-SA 4.0
2023-09-26T20:52:22.727Z
@Oddthinking: depending where you're coming from, mypy either provides "sophisticated static types and analysis", or else it provides decidedly *unsophisticated* static types and analysis. I use it, and it's fine, but it doesn't exactly make Haskell programmers sing with joy when they see it. Likewise there are various things that you'd routinely do in C++ for TMP, that just aren't available in mypy.
Steve Jessop
CC BY-SA 4.0
2023-09-26T21:23:42.010Z
@kaya3: Unmanaged memory is not a particularly great example of a resource with an associated responsibility, since unmanaged heap storage is generally fungible.  So long as they get freed eventually, timeliness won't usually matter.  Things like files and locks are a much bigger problem.  Code which fails to close a file may block execution of another program that would need it; no matter how many other files or how much disk space a system has, a program that needs exclusive access to one particular file will be blocked as long as that file is open somewhere else.
supercat
CC BY-SA 4.0
2023-09-26T21:25:35.660Z
@kaya3: I agree with your point about GC-based languages failing to provide programmers with the right tools to handle resources/responsibilities, but simply don't think unmanaged memory is the best example of resources encapsulating responsibilities.
supercat
CC BY-SA 4.0
2023-09-26T21:46:02.847Z
@supercat Well, it's the example that is pertinent to this question.
kaya3
CC BY-SA 4.0
2023-09-26T21:56:10.223Z
@kaya3: If a language has good support for managing responsibilities in general, such features will also be a practical way of handling unmanaged memory.  While there are some approaches like finalizers which may be suitable for wrangling unmanaged memory, but not for fungible resources, the most practical way to facilitate working with unmanaged memory would be to facilitate working with resources in general.
supercat
CC BY-SA 4.0
2023-09-26T22:14:47.093Z
@supercat Nonetheless, it is a key problem which needs to be overcome, and that's what the question asks about.
kaya3
CC BY-SA 4.0
2023-09-26T23:42:29.220Z
An example where cleaning up resources "eventually" is not sufficient, but the resources are fungible, is sockets. Suppose you have a limit on total open file descriptors per process, and you don't close sockets properly but leave them to GC. You run out of file descriptors long before you run out of memory. So yeah, why does the questioner want to mix GC and non-GC memory? Does that desire suggest their resource management in general is broken? If their resource management in general is sound, they don't strictly need "unmanaged" memory, they need a resource manager *for* memory ;-)
Steve Jessop
<p>Embedded developer here. Yes, it is possible and, yes, there are real-world examples. C# is a particularly common one.</p>
<p>The vast majority of memory allocations (and other resource allocations) in C# are managed (i.e. garbage collected,) but you can use manually-allocated memory as well as unmanaged resources allocated from other operating system APIs when you need to. The vast majority of C# applications never need to do this, but, when you have performance-critical code or just need to interface with some API that uses unmanaged resources, you can manually allocate memory and use pointers for it just as you would in C or C++.</p>
<p>One of the most useful aspects of this in C# in particular is that it allows you to access an array without bounds checking being performed automatically by the runtime. While bounds checking is great in most cases, when you're processing an extremely large amount of data, omitting it will result in vastly faster execution time. This is especially useful in the embedded world where you typically don't have the CPU power of a desktop computer and you're frequently running on battery power, but it can be useful in any application that needs to process very large data sets. Of course, with no automatic bounds checking, it is up to the developer to make sure not to read or write outside of the buffer's bounds and attempting to do so results in undefined behavior, just as it would in C or C++.</p>
<p>Of course, for managed memory, the system can garbage collect or even relocate a buffer at completely unpredictable times, so just grabbing a pointer to a managed buffer and then using it later results in undefined behavior. In order to resolve this problem, C# provides the <a href="https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/fixed" rel="noreferrer"><code>fixed</code> statement</a>. This statement creates a scope within which the specified managed array is not eligible for garbage collection and cannot be relocated by the garbage collector and provides a pointer for direct access to that array's memory within that scope. Using the pointer outside of that scope is an invalid operation and will result in undefined behavior, but this is no different from attempting to use a buffer after deallocating it in C or C++. You just have to ensure that your code does not use the pointer again outside of the <code>fixed</code> statement.</p>
<p>As <a href="https://langdev.stackexchange.com/a/2957/2581">kaya3's answer</a> discussed, another problem with mixing managed and unmanaged resources is dealing with the case where a managed object references an unmanaged resource and the managed object is garbage collected. The way C# (and .Net in general) handles this is the <a href="https://learn.microsoft.com/en-us/dotnet/standard/design-guidelines/dispose-pattern" rel="noreferrer">Dispose Pattern</a>. In the Dispose pattern, there a managed object which might have unmanaged resources to clean up (or which might need to dispose other managed objects) has a public <code>Dispose()</code> method, which calls a protected <code>Dispose(bool)</code> with the Boolean parameter set to <code>true</code>. If it has unmanaged resources to clean up, it also implements a finalizer, which calls <code>Dispose(false)</code>. The <code>Dispose(bool)</code> method always cleans up unmanaged resources, but only attempts to dispose other managed resource if called with its Boolean parameter set to true, since those resources may have already been garbage collected if <code>Dispose(bool)</code> was called from the finalizer. As long as this pattern is followed correctly, unmanaged resources owned by managed objects will be released when the program is done with the managed object, even if <code>Dispose()</code> is never called explicitly on it.</p>
<p>As an embedded developer who writes everything from WPF touchscreen user interfaces down to PCIe device drivers, personally I love having the flexibility to write the vast majority of our user-mode applications in a managed language, but still be able to use pointers when I need to crunch through a very large dataset efficiently or interoperate with libraries written in other language, such as C. 95% or so of our code can still benefit from the ease and safety of managed memory without having to compromise performance on the other 5% or else write the other 5% in a different language. And it also means that we can create managed objects to 'wrap' unmanaged resources in exactly the same way that the runtime itself does this for operating system resources such as file handles, mutexes, etc.</p>

CC BY-SA 4.0
2023-09-25T22:47:56.547Z
2023-09-25T23:48:17.003Z
2023-09-25T23:48:17.003Z
reirab
CC BY-SA 4.0
2023-09-26T17:29:11.723Z
As an embedded developer, I find it interesting that many people associate GC systems with bigger platforms, but the first really popular language implementations to use GC were designed for machines with as little as 4K o4 5K of RAM.  The GC algorithm used was terriblely slow, but it did manage a tiny memory footprint.
supercat
CC BY-SA 4.0
2023-10-05T19:27:55.437Z
Slight technicality,  but most c# applications do need to do this somewhere, it's most c# code that doesn't.  Most c# developers will only ever use unmanaged capabilities through libraries (including the std lib) but it still comes up on profilers etc.
user1937198
CC BY-SA 4.0
2023-10-06T01:20:34.953Z
@user1937198 Yes, that's definitely true. I really meant that the code written by the application developer in the vast majority of C# applications would never need to do that, not that the libraries weren't doing it under the hood. The runtime and standard libraries are, of course, filled with pointers or handles to unmanaged resources of all sorts, but, at least in the case of the ones in the libraries, they use that same Dispose Pattern to wrap those in managed objects so that application developers won't need to deal with the unmanaged resources.
reirab
<p>Yes, a very loose summary of a system I worked with (and implementing) pre-2007:</p>
<p>The main part was an assembly language designed with a view to implementing Java. Or maybe refined for Java, I don't remember: that part happened before I joined the company. Think of an assembly language with a lot of macro facilities built in, including macros that make virtual function calls and some other relatively high-level concepts. All this targeted a virtual instruction set that would later be translated to actual machine code (not a million miles from the idea of LLVM instructions, but invented before LLVM itself existed).</p>
<p>So, all essential JVM opcodes were available as assembler macros or function calls. At the same time, by which I mean, from the same source code written in the same language inside the same function as these Java operations, the programmer also had direct access to the kernel calls of its non-protected mode OS. And also to a stdlib style set of functions. The stdlib was mostly used by the system's C compiler when it generated the backend code, but there's nothing to stop anyone calling them who is writing in assembly.</p>
<p>If you stored a pointer to a Java object into a marked region of memory (the stack or a Java object, basically) then it would be GCed normally (normally for Java, I mean: mark sweep). You could also allocate unmanaged memory just as easily with <code>malloc</code> and <code>free</code> themselves, or with similar-style kernel functions. I'm pretty sure there was also some facility to help with reference counting, although I don't remember exactly how that worked. Everything you can do in Java translated reasonably obviously to assembly code, it just took longer to write.</p>
<p>So, when writing code you genuinely made a choice for each piece of memory whether to allocate it as a Java object (in which case it would be GCed provided you didn't mess up and fail to store the pointer anywhere), from the kernel (in which case you were responsible for freeing it, via the assisted refcounting if you wanted it), or from libc (in which case you were <em>broadly</em> responsible for freeing it, but as a concession and to support easier porting of existing C programs, there was a concept of a process, and processes did keep records of all their <code>malloc</code> allocations and other C-style resources, to clean up on process exit).</p>
<p>It therefore had more mixed modes of resource management than any one piece of code really wanted to use: you'd mix at most two of the three in practice.</p>
<p>This system was never publicly available as a programming environment, although as a Java implementation it did appear in some embedded devices, including early HTC Java phones. It also made it onto some other devices without the Java parts: it was quite modular in that sense and you could build a stripped-down version of the OS that just didn't include anything you didn't call.</p>
<p>And fundamentally, it did what you ask for but probably not what you want. It didn't have the &quot;ease of use&quot; of Java, because you literally had to think, &quot;OK, is my next virtual call made through an interface or through a class&quot;, because invoke_virtual and invoke_interface were different macros. It had a language restriction that each instruction had at most one side-effect (counting &quot;call out&quot; as one side-effect): you could write <code>a + b + c</code> if those were integers, but you literally couldn't write <code>a() + b() + c()</code> if those were function pointers. You couldn't even write <code>total += a()</code>. You had to write the equivalent of <code>result = a(); total += result</code>. You couldn't write <code>a + b + c</code> if those were strings, either: it would be probably 4 lines of code to create a StringBuilder from <code>a</code>, then append <code>b</code>, then append <code>c</code>, then create a string from the result. When you did that, all that memory would be GCed. But the assembler absolutely did not type check your code: but you didn't ask for type checking, you asked for mixed memory management ;-)</p>
<p>One sense it probably didn't do what you're asking: you could mix the modes, but not for the same types. If you want a GCed array of integers, fine, use a Java array (and of course you could get the base address to do direct memory access to it as a buffer). But if you want to manually manage a Java object, you can't: there was no legitimate way to call &quot;free&quot; on a java.lang.String on grounds you happen to know you're the last remaining reference to it. If you want to do that you need to allocate a buffer for a nul-terminated byte array from the kernel, instead, and then it wouldn't have all the convenience methods of String. Hello, <code>strncpy</code>.</p>
<p>So there's a reason it was never intended as <strong>the</strong> new general-purpose programming language for the world. But for relatively high-end (for the time) embedded systems programming, and for accelerating Java code before Sun ever released a JIT for mobile devices: it wasn't bad. We beat Sun on pretty much all the benchmarks at the time. You could take Java code, figure out which bits &quot;really needed&quot; GC and which could be manually managed, and rewrite the Java code to run faster and occupy less RAM. We literally did that for some critical parts of the Java standard libraries, and left other parts as the off-the-shelf Java implementations.</p>

CC BY-SA 4.0
2023-09-26T21:22:18.693Z
2023-09-26T22:00:06.933Z
2023-09-26T22:00:06.933Z
Steve Jessop
CC BY-SA 4.0
2023-09-26T21:33:59.703Z
Aside from all this complicated stuff, there's also things like the Boehm GC for C code. It uses gc "by default" because you call `GC_MALLOC` "by default" instead of `malloc`. But you can still manually manage memory by calling `malloc` and `free` instead. That's what you asked for, right? ;-)
Steve Jessop
CC BY-SA 4.0
2023-09-26T22:23:33.603Z
Welcome, this was really interesting to read! Given this answer and another one, perhaps embedded/mobile development is a niche where this kind of "mixing" of memory management strategies makes sense and is worth it?
kaya3
CC BY-SA 4.0
2023-09-26T22:54:59.170Z
@kaya3: certainly I'd say that embedded is a realm where developers are more likely to want to write code that looks like systems programming, even though what you're actually writing is definitely an application. Of course, "mobile" today means a device as powerful or more than the desktop I was using in 2000 to write this code for what we then called smartphones, and now would call barely fit to use as a paperweight.
Steve Jessop
CC BY-SA 4.0
2023-09-26T23:00:56.053Z
I was thinking also that perhaps FFI is less of an option in (some) embedded contexts; if you can't farm out the manual memory management to a separate lower-level language then it makes more sense to have it alongside GC-managed code in the same language. But this is just my speculation since I have no experience with embedded programming.
kaya3
CC BY-SA 4.0
2023-09-26T23:02:39.953Z
Yeah, although I think probably that model does also work very well: you "just" need the high-level language to have a reasonably lightweight interpreter. I don't do embedded any more and I don't know the state of the art, but even back then there was always a tension between the code that you want to write quickly, and the code that you want to hand-optimise. You didn't want to write everything in assembly if you could avoid it: but at the time we considered even C kind of heavyweight unless you were very careful what std libraries you called.
Steve Jessop
CC BY-SA 4.0
2023-09-26T23:06:42.673Z
Also, sad to say, at the time we usually got genuinely better code out of hand-written assembly than out of GCC, even from just "bashing out" the assembly the obvious way, before trying to optimise. There was just always something GCC felt obliged to keep track of that you didn't really need, like it missed a zap that should be obvious, or didn't reuse stack, or just something. I expect that has probably changed now, and you have to be really quite good at assembly to beat GCC without trying. Likely this was not entirely GCC's fault, and our GCC backend could have been better too.
Steve Jessop
CC BY-SA 4.0
2023-09-26T23:26:08.377Z
JNI is at least in the same neighbourhood too. Absolutely it's an FFI between Java and C. But if you choose to ignore the Java programming language, write all your code in C, and third parties send you bytecode to execute along with it; then that's the general idea. Down in your JNI code you can choose between allocating C objects to manage yourself, or allocating Java objects to be GCed. We effectively had JNI with zero change of context, because we were writing in the assembly for the same instruction set that Java bytecode got translated to when loaded. There was no Java interpreter.
Steve Jessop
CC BY-SA 4.0
2023-09-29T21:56:44.550Z
//I expect that has probably changed now, and you have to be really quite good at assembly to beat GCC without trying.//  That's certainly not true for the Cortex-M0.  Sometimes it seems the only way to coax clang and gcc to generate good code for a loop is to use `volatile` to prevent gcc from replacing an add-to-register instruction within a loop with a combination of load-constant-into-register instruction, move register, and add-to-register instruction (all within the loop), or prevent clang from transforming something like `while (x >= 0)` into `while (x > (stepsize-1))`.
supercat
<p>An abstraction model which uses explicit object disposal will be essentially as good or better than garbage collection in most usage cases, but at least three usage cases where garbage collection is better are biggies:</p>
<ol>
<li><p>It's useful to be able to pass around references to immutable data-holding objects, including objects of mutable type which are wrapped by immutable objects and will never change after wrapper construction is complete, as proxies for the data contained therein, without having to know or care how many references exist to any particular object.</p>
</li>
<li><p>Sometimes it's useful for objects to perform actions on behalf of other objects, without having any particular interest in those objects.  For example, an object might notify an &quot;updates counter&quot; object every time it's updated, but if the last entity that's interested in the update counter goes away, the monitored object's reference to the update counter should go away at some point(*)</p>
</li>
<li><p>A tracing garbage-collector can ensure that after an object is notified its services will no longer be required, notifies other entities upon which it relies likewise, and consequently becomes useless, references to that object will continue to remain as valid references to a useless object, and the object can then deterministically reject attempts to continue using it rather than having such requests act upon other arbitrary objects, with unpredictable effects.</p>
</li>
</ol>
<p>The existence of a garbage collector in no way obviates the need for explicit deterministic resource cleanup, but even the best imaginable mechanisms for explicit resource cleanup in no way obviate the above advantages for a tracing garbage collector.</p>
<p>(*) I've yet to see a framework handle this particularly well; a design I'd like to see would be to have event handlers provide an &quot;are you still interested?&quot; function which would allow an event handler to indicate whether further notifications were required.  If the event handler has been told its services are no longer required, it could let any object that sends it an event know (via return value) that it should be unsubscribed.  To guard against memory leaks, events with event lists could send &quot;are you still interested?&quot; messages to two subscribers (if that many exist) every time a new subscription is added.  Unused subscription entries for dead objects might remain around indefinitely, but there would be no way for subscription lists to fill up with an unbounded number of such entries, since old once would be getting cleaned out while new ones were added.</p>

CC BY-SA 4.0
2023-09-26T15:19:05.803Z
2023-09-26T15:32:15.767Z
2023-09-26T15:32:15.767Z
supercat
CC BY-SA 4.0
2023-09-27T02:46:37.757Z
(2) sounds a bit like the "entity that's interested in the update counter" should hold a hard reference, and the "monitored object" should hold a weak reference. Then, when the last entity that's interested in the update counter goes away, the monitored object's reference to the update counter can no longer be acquired - this failure to acquire could be interpreted the same way you'd interpret a negative response to "are you still interested?". Of course weak references have a non-trivial implementation overhead. Am I missing something else, or is the problem that cost?
Steve Jessop
CC BY-SA 4.0
2023-09-27T15:51:31.437Z
@SteveJessop: It's possible to manage views using ownership and "non-owning" pointers, even without GC, but that requires adding another layer of notifications to deal with changes to subscriptions, and all interactions among notifications need to be handled in thread-safe and deadlock-free fashion.  If one has an object which is e.g. supposed to count how many times some monitored event happens, but not do anything other than hold the count, allowing multiple objects who are interested in the thing being watched to share that object without the observer having to care can simplify things.
supercat
<p>Yes</p>
<p>Qt, when used with C++, does this.
The Qt data structures are managed. However, nothing prevents you from using C++ or even plain old C pointers at the same time.
One could even use Qt and STL data structures at the same time. For example, QVector and std::vector. Can be useful when interfacing with libraries which don't use Qt.</p>

CC BY-SA 4.0
2023-09-27T12:16:46.460Z
2023-09-27T12:16:46.460Z
vsz
<p>Yes.</p>
<p>Object Pascal behaves like that. Class instances and ordinary blocks of memory are subject to manual allocation and deallocation, while strings and dynamic arrays are reference-counted.</p>

CC BY-SA 4.0
2023-09-26T06:50:32.053Z
2023-09-26T06:50:32.053Z
Mark Morgan Lloyd
# Possible to mix garbage collection and manual memory management?
Posted by: username_0 on 2023-09-24
Do you think it is possible to have a language that uses garbage collection (GC) by default, but allows you take more control with manual memory management like C++ or Rust, in areas of the software where you care more about performance?

Are there any examples of languages that do this, in academia or the "real world"? Are there particular key problems that need to be overcome?

Maybe the allocation strategy could be a generic type parameter.
**Tags:** garbage-collection memory-management
> **Comments:**
> **username_1**: “Are there any examples of languages that do this”: Nim.  
> **username_2**: Another example: early Rust.  
> **username_3**: C# also does that.  
> **username_4**: Expanding on @MartinBerger's comment: early Rust used to have a garbage collector for the things it couldn't verify statically.  
> **username_5**: Conservative GC has been bolted on to languages with manual memory management.  
> **username_6**: Why are you Asking here whether there languages that do this, in academia or the 'real world'? Why are you not Posting your own research?  
> **username_7**: There is a new language that is under development, that allows you to specify the GC mechanism as a compile-time option :  [Vlang](https://vlang.io/), there is also a third option which also emits the necessary deallocation code at the compile-time.  
> **username_8**: Swift, which uses Reference Counting GC, is another example of a language that also supports dealing with [manual memory management](https://developer.apple.com/documentation/swift/manual-memory-management), mostly for interfacing with C APIs.  
> **username_9**: Just to be clear, what about Python makes it not match your criteria? It uses gc by default, but if you don't allow any references to your object to "escape", then CPython guarantees that it will use reference-counting, and that (if there are no reference loops) your object *will* be destroyed on exit from the scope from which it doesn't escape. It doesn't have a "delete" instruction that can destroy an object despite it still being referenced, but it does have *some* built-in control of object lifetime. How much more control does a language need to provide, in order to qualify?  
> **username_9**: I mean, there's also a ton of other reasons not to use Python for the code where you absolutely positively care most about performance. But this isn't a question about the GIL or about real-time guarantees, it's just about avoiding gc. You could say reference-counting is still GC, but then any language that uses reference-counting by default, falls back to manual, and has no way to even clean up reference loops, would qualify. ISTR Perl maybe in that state at some early stage, before mark-sweep was implemented.  
> **username_0**: @SteveJessop To your question of why Python doesn't work for me... It might, but it's not ideal. I work on apps with pieces that need high performance, mostly around graphics, computation geometry, math. I haven't found anything that beats C or C++ here, for me. But to write a whole app in it is painful. I also have a vision of building software with an IDE that is more like ...  
> **username_0**: ... (cont) a notebook environment - interactive, literate programming (Like Matlab, Jupyter) so you intermix code,  documentation, example output. I might have to use Python, but I also really like advanced type systems and trying to prove code works, and on this Python went the other direction.  
> **username_9**: There is the "I like notebooks" guy, Jeremy Howard, who does everything in notebooks and nbdev, including developing, documenting and building entirely non-graphical libraries for use by people whether they're in Jupyter or not. I still can't tell whether I think he's right or just ignoring a heap of problems, and I'm not brave enough to find out for myself. FFI is totally the dominant paradigm for performance in the Python ecosystem, though. Stuff like numpy gets approx 0% of its performance gain from avoiding GC, and approx 100% from avoiding allocations and Python bytecodes entirely.  
> **username_0**: Thanks. I may end up using that. I've been messing with creating my own language that is interactive, but with types, a JIT, Swift-like syntax. It may just be too crazy hard to do. So for my sanity I may have to drop it. // About the perf gains of memory control, I doubt it's 0%, but Python is so slow the other issues dominate. For some things I found C++ beat Swift. I'm not sure why, but one of the only differences there is more control over memory allocation, and Swift using a lot of reference counting.  
> **username_10**: Is it _possible?_ Of course! The trivial way to do it is to have two completely separate heaps: One with GC and one without.  
> **username_11**: „one of the only differences there is more control over memory allocation“ – … and the fact that the Swift compiler has been optimized for 13 years by one company whereas C++ compilers have been optimized for 44 years by dozens of companies … including Apple. True, Swift also benefits from all the work poured into LLVM, but then again, so do Rust, Kotlin, Fortran, Ada, D, Delphi, Go, C, and of course C++  
---
## 6 answers
### Answer by username_17 on 2023-09-25. Score: 15
Embedded developer here. Yes, it is possible and, yes, there are real-world examples. C# is a particularly common one.

The vast majority of memory allocations (and other resource allocations) in C# are managed (i.e. garbage collected,) but you can use manually-allocated memory as well as unmanaged resources allocated from other operating system APIs when you need to. The vast majority of C# applications never need to do this, but, when you have performance-critical code or just need to interface with some API that uses unmanaged resources, you can manually allocate memory and use pointers for it just as you would in C or C++.

One of the most useful aspects of this in C# in particular is that it allows you to access an array without bounds checking being performed automatically by the runtime. While bounds checking is great in most cases, when you're processing an extremely large amount of data, omitting it will result in vastly faster execution time. This is especially useful in the embedded world where you typically don't have the CPU power of a desktop computer and you're frequently running on battery power, but it can be useful in any application that needs to process very large data sets. Of course, with no automatic bounds checking, it is up to the developer to make sure not to read or write outside of the buffer's bounds and attempting to do so results in undefined behavior, just as it would in C or C++.

Of course, for managed memory, the system can garbage collect or even relocate a buffer at completely unpredictable times, so just grabbing a pointer to a managed buffer and then using it later results in undefined behavior. In order to resolve this problem, C# provides the [`fixed` statement](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/fixed). This statement creates a scope within which the specified managed array is not eligible for garbage collection and cannot be relocated by the garbage collector and provides a pointer for direct access to that array's memory within that scope. Using the pointer outside of that scope is an invalid operation and will result in undefined behavior, but this is no different from attempting to use a buffer after deallocating it in C or C++. You just have to ensure that your code does not use the pointer again outside of the `fixed` statement.

As [kaya3's answer](https://langdev.stackexchange.com/a/2957/2581) discussed, another problem with mixing managed and unmanaged resources is dealing with the case where a managed object references an unmanaged resource and the managed object is garbage collected. The way C# (and .Net in general) handles this is the [Dispose Pattern](https://learn.microsoft.com/en-us/dotnet/standard/design-guidelines/dispose-pattern). In the Dispose pattern, there a managed object which might have unmanaged resources to clean up (or which might need to dispose other managed objects) has a public `Dispose()` method, which calls a protected `Dispose(bool)` with the Boolean parameter set to `true`. If it has unmanaged resources to clean up, it also implements a finalizer, which calls `Dispose(false)`. The `Dispose(bool)` method always cleans up unmanaged resources, but only attempts to dispose other managed resource if called with its Boolean parameter set to true, since those resources may have already been garbage collected if `Dispose(bool)` was called from the finalizer. As long as this pattern is followed correctly, unmanaged resources owned by managed objects will be released when the program is done with the managed object, even if `Dispose()` is never called explicitly on it.

As an embedded developer who writes everything from WPF touchscreen user interfaces down to PCIe device drivers, personally I love having the flexibility to write the vast majority of our user-mode applications in a managed language, but still be able to use pointers when I need to crunch through a very large dataset efficiently or interoperate with libraries written in other language, such as C. 95% or so of our code can still benefit from the ease and safety of managed memory without having to compromise performance on the other 5% or else write the other 5% in a different language. And it also means that we can create managed objects to 'wrap' unmanaged resources in exactly the same way that the runtime itself does this for operating system resources such as file handles, mutexes, etc.
> **Comments:**
> **username_15** (2023-09-26): As an embedded developer, I find it interesting that many people associate GC systems with bigger platforms, but the first really popular language implementations to use GC were designed for machines with as little as 4K o4 5K of RAM.  The GC algorithm used was terriblely slow, but it did manage a tiny memory footprint.  
> **username_18** (2023-10-05): Slight technicality,  but most c# applications do need to do this somewhere, it's most c# code that doesn't.  Most c# developers will only ever use unmanaged capabilities through libraries (including the std lib) but it still comes up on profilers etc.  
> **username_17** (2023-10-06): @user1937198 Yes, that's definitely true. I really meant that the code written by the application developer in the vast majority of C# applications would never need to do that, not that the libraries weren't doing it under the hood. The runtime and standard libraries are, of course, filled with pointers or handles to unmanaged resources of all sorts, but, at least in the case of the ones in the libraries, they use that same Dispose Pattern to wrap those in managed objects so that application developers won't need to deal with the unmanaged resources.  
---
### Answer by username_12 on 2023-09-24. Score: 11
Certainly it is possible, but do people actually want it?

Well, of course we all want "the best of both worlds". But by and large, each world is big enough by itself to satisfy almost everyone who lives there: languages with manual memory management tend to have standard library or third-party library features for some levels of automatic memory management (e.g. [reference counting](https://en.wikipedia.org/wiki/Reference_counting) and [arena allocation](https://en.wikipedia.org/wiki/Region-based_memory_management) cover most use-cases); and the performance cost of having a GC for every object is clearly acceptable to many programmers, and often negligible in practice.

> Are there particular key problems that need to be overcome?

The problems will occur when you try to mix the two. If a program uses exclusively managed objects, or exclusively unmanaged memory allocations, then there is no problem, but in that case there would also be no reason to use a language which supports both ─ you might as well just have two different languages.

It's worth noting that many of these problems also apply to [FFI](https://en.wikipedia.org/wiki/Foreign_function_interface), for example when writing C extensions for Python. This isn't exactly mixing managed and unmanaged memory in the same language, but the C programmer who writes the extension does have to account for Python's garbage collection. The upside is that Python programmers can't accidentally cause problems with unmanaged memory.

So if you want the benefits of a GC'd language but you also want to offer careful programmers the performance benefits of having some manually managed allocations, consider FFI as an alternative language design feature.

## Managed objects holding references to unmanaged memory

A garbage-collected object which holds a reference to an unmanaged allocation, will have to manually deallocate the unmanaged memory when the managed object is collected. You can do this with [finalisers](https://en.wikipedia.org/wiki/Finalizer), but there are [many problems associated with finalisers](https://en.wikipedia.org/wiki/Finalizer#Problems):

* They're called non-deterministically in an unreliable order, so you can't guarantee anything about the program state (e.g. object invariants) when they're invoked;
* They are invoked by the GC but can change the live-status of other objects, complicating the GC implementation;
* Programmers frequently misuse them for non-memory resources;
* There's no natural place in the program to catch or handle exceptions or other runtime errors raised in finalisers;
* If the GC is multithreaded then finalisers need to be written to take care of this, even if the program itself is single-threaded; deadlocks in finalisers are also possible.

These are all reasons that newer languages tend to avoid having finalisers, and older languages are deprecating them (e.g. [Java](https://stackoverflow.com/q/56139760/12299000)). But without finalisers, or something like them, GC-managed objects can't own unmanaged memory without leaking it when they're collected.

An alternative to finalisers which solves some problems is to have the GC enqueue notifications about collected objects (i.e. [phantom references](https://en.wikipedia.org/wiki/Phantom_reference)) into a user-controlled queue, so that the user's program can choose when to traverse that queue and in which thread; this should be safer overall, but it shifts more work onto the user, and that comes with new potential sources of bugs.

## Unmanaged memory holding references to managed objects

A manually-managed allocation which holds a reference to a GC-managed object will have to manually inform the GC about it. The GC needs to know about all roots (definitely-live references to managed objects); it can know automatically about roots held by the call stack, but not those held in unmanaged memory. Then, when the unmanaged allocation is deallocated, the programmer needs to again manually inform the GC that the reference has been dropped.

This basically means the programmer must manually write code which would otherwise be generated automatically by a tracing GC.

## Raw memory access of managed objects

If the language allows reading/writing to memory using raw pointers, then the language must ensure that either

* Nobody can acquire a pointer to a managed object;
* Managed objects have reliable, specified memory layouts (in particular, the object header used by the GC must be a fixed or reliably computed size); or
* Otherwise, if programmers can directly read/write to managed allocations, that they know not to do this (although realistically, they will do it anyway).

The issue is that writing to an object header will generally lead to [undefined behaviour](https://en.wikipedia.org/wiki/Undefined_behaviour) in the sense that anything could happen. The GC is a black box, so a mutated object header could have arbitrary meaning to the GC; a mutated header might violate some invariants, or cause a finaliser to be invoked at the wrong time.
> **Comments:**
> **username_0** (2023-09-24): Very useful answer, thanks. Though I'm not sure I agree that there is "no reason" to use such a language. I think I would like it. ... Good point that a one-language solution would be a lot like an FFI. I've been tinkering with creating my own language. Python *almost* works for me, because I want interactive/notebook development and I can work with their two-language/FFI solution to performance. My problem with Python is the lack of sophisticated static types and analysis.  
> **username_12** (2023-09-24): @RobN The "no reason" part is if you only ever use one or the other ─ that is, there are problems if you do use both features together, those problems don't occur if you don't mix them but there's no point having a language that lets you mix them if you always avoid doing so.  
> **username_13** (2023-09-25): There are other options besides finalizers. Things like [PhantomReference](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ref/PhantomReference.html) and reference queues, for example. They don't avoid *all* the problems of finalizers, but they avoid a lot - for example, a PhantomReference can't resurrect objects.  
> **username_14** (2023-09-25): @RobN: "My problem with Python is the lack of sophisticated static types and analysis." Have you looked at [mypy](https://mypy-lang.org/)?  
> **username_11** (2023-09-25): @user2357112: [*Ephemerons*](https://wikipedia.org/wiki/Ephemeron) sound weakly related (pun intended) as well.  
> **username_15** (2023-09-25): Instead of referring to "unmanaged memory" I think it is more useful to think of resources in terms of *responsibilities* that are acquired, and may only be released by fulfilling them or handing them off.  Code that opens a file acquires a responsibility to close it; code that acquires a lock also acquires a responsibility to release it, etc.  An advantage of a hybrid approach where code manually carries out responsibilities is that the GC can ensure that a reference to an object associated with e.g. a file, will always be a valid reference to something, even if it's an object that's...  
> **username_15** (2023-09-25): ...unable to do anything except that it's a permanently useless object.  Also, GC frameworks may allow objects to be pinned, with a proviso that code must ensure that no outside copies of the address will ever be used after an object is unpinned.  
> **username_16** (2023-09-26): I strongly agree with supercat's "responsibilities" framework. When you allocate an unmanaged object, you assume responsibility for its disposal. Some of the objections discussed in your answer represent cases where the programmer is not living up to this responsibility. (Most clearly "unmanaged object owned by a managed object.") You shouldn't (I claim) lay blame for this on the _mixture_ of modes; it's already a consequence of using unmanaged memory at all.  
> **username_12** (2023-09-26): @GlennWillen The "responsibilities" model does not contradict anything written in my answer; indeed, an object has a responsibility to dispose of any resources it owns, including unmanaged memory, so therefore you need something like a finaliser if a managed object owns an unmanaged allocation. I'm not saying it's a problem that there are responsibilities at all; I'm saying there is a problem when a GC-managed object represents such a responsibility. Therefore this problem *is* specifically associated with mixing managed and unmanaged memory. The same issue occurs when a GC-managed object ...  
> **username_12** (2023-09-26): ... some other resource such as a file handle or network socket; the OOP model of representing things with GC-managed objects is not a good fit for managing responsibilities. Again, it is not fundamentally a problem to *have* resources which create a responsibility to dispose of them, the problem is when you mix that with a paradigm that is not well-suited for disposing of such resources deterministically at definite times. Yes, this means the programmer is not living up to the responsibility, but that particularly happens when the language gives the wrong tools for the programmer to do so.  
> **username_9** (2023-09-26): @Oddthinking: depending where you're coming from, mypy either provides "sophisticated static types and analysis", or else it provides decidedly *unsophisticated* static types and analysis. I use it, and it's fine, but it doesn't exactly make Haskell programmers sing with joy when they see it. Likewise there are various things that you'd routinely do in C++ for TMP, that just aren't available in mypy.  
> **username_15** (2023-09-26): @kaya3: Unmanaged memory is not a particularly great example of a resource with an associated responsibility, since unmanaged heap storage is generally fungible.  So long as they get freed eventually, timeliness won't usually matter.  Things like files and locks are a much bigger problem.  Code which fails to close a file may block execution of another program that would need it; no matter how many other files or how much disk space a system has, a program that needs exclusive access to one particular file will be blocked as long as that file is open somewhere else.  
> **username_15** (2023-09-26): @kaya3: I agree with your point about GC-based languages failing to provide programmers with the right tools to handle resources/responsibilities, but simply don't think unmanaged memory is the best example of resources encapsulating responsibilities.  
> **username_12** (2023-09-26): @supercat Well, it's the example that is pertinent to this question.  
> **username_15** (2023-09-26): @kaya3: If a language has good support for managing responsibilities in general, such features will also be a practical way of handling unmanaged memory.  While there are some approaches like finalizers which may be suitable for wrangling unmanaged memory, but not for fungible resources, the most practical way to facilitate working with unmanaged memory would be to facilitate working with resources in general.  
> **username_12** (2023-09-26): @supercat Nonetheless, it is a key problem which needs to be overcome, and that's what the question asks about.  
> **username_9** (2023-09-26): An example where cleaning up resources "eventually" is not sufficient, but the resources are fungible, is sockets. Suppose you have a limit on total open file descriptors per process, and you don't close sockets properly but leave them to GC. You run out of file descriptors long before you run out of memory. So yeah, why does the questioner want to mix GC and non-GC memory? Does that desire suggest their resource management in general is broken? If their resource management in general is sound, they don't strictly need "unmanaged" memory, they need a resource manager *for* memory ;-)  
---
### Answer by username_9 on 2023-09-26. Score: 5
Yes, a very loose summary of a system I worked with (and implementing) pre-2007:

The main part was an assembly language designed with a view to implementing Java. Or maybe refined for Java, I don't remember: that part happened before I joined the company. Think of an assembly language with a lot of macro facilities built in, including macros that make virtual function calls and some other relatively high-level concepts. All this targeted a virtual instruction set that would later be translated to actual machine code (not a million miles from the idea of LLVM instructions, but invented before LLVM itself existed).

So, all essential JVM opcodes were available as assembler macros or function calls. At the same time, by which I mean, from the same source code written in the same language inside the same function as these Java operations, the programmer also had direct access to the kernel calls of its non-protected mode OS. And also to a stdlib style set of functions. The stdlib was mostly used by the system's C compiler when it generated the backend code, but there's nothing to stop anyone calling them who is writing in assembly.

If you stored a pointer to a Java object into a marked region of memory (the stack or a Java object, basically) then it would be GCed normally (normally for Java, I mean: mark sweep). You could also allocate unmanaged memory just as easily with `malloc` and `free` themselves, or with similar-style kernel functions. I'm pretty sure there was also some facility to help with reference counting, although I don't remember exactly how that worked. Everything you can do in Java translated reasonably obviously to assembly code, it just took longer to write.

So, when writing code you genuinely made a choice for each piece of memory whether to allocate it as a Java object (in which case it would be GCed provided you didn't mess up and fail to store the pointer anywhere), from the kernel (in which case you were responsible for freeing it, via the assisted refcounting if you wanted it), or from libc (in which case you were *broadly* responsible for freeing it, but as a concession and to support easier porting of existing C programs, there was a concept of a process, and processes did keep records of all their `malloc` allocations and other C-style resources, to clean up on process exit).

It therefore had more mixed modes of resource management than any one piece of code really wanted to use: you'd mix at most two of the three in practice.

This system was never publicly available as a programming environment, although as a Java implementation it did appear in some embedded devices, including early HTC Java phones. It also made it onto some other devices without the Java parts: it was quite modular in that sense and you could build a stripped-down version of the OS that just didn't include anything you didn't call.

And fundamentally, it did what you ask for but probably not what you want. It didn't have the "ease of use" of Java, because you literally had to think, "OK, is my next virtual call made through an interface or through a class", because invoke\_virtual and invoke\_interface were different macros. It had a language restriction that each instruction had at most one side-effect (counting "call out" as one side-effect): you could write `a + b + c` if those were integers, but you literally couldn't write `a() + b() + c()` if those were function pointers. You couldn't even write `total += a()`. You had to write the equivalent of `result = a(); total += result`. You couldn't write `a + b + c` if those were strings, either: it would be probably 4 lines of code to create a StringBuilder from `a`, then append `b`, then append `c`, then create a string from the result. When you did that, all that memory would be GCed. But the assembler absolutely did not type check your code: but you didn't ask for type checking, you asked for mixed memory management ;-)

One sense it probably didn't do what you're asking: you could mix the modes, but not for the same types. If you want a GCed array of integers, fine, use a Java array (and of course you could get the base address to do direct memory access to it as a buffer). But if you want to manually manage a Java object, you can't: there was no legitimate way to call "free" on a java.lang.String on grounds you happen to know you're the last remaining reference to it. If you want to do that you need to allocate a buffer for a nul-terminated byte array from the kernel, instead, and then it wouldn't have all the convenience methods of String. Hello, `strncpy`.

So there's a reason it was never intended as **the** new general-purpose programming language for the world. But for relatively high-end (for the time) embedded systems programming, and for accelerating Java code before Sun ever released a JIT for mobile devices: it wasn't bad. We beat Sun on pretty much all the benchmarks at the time. You could take Java code, figure out which bits "really needed" GC and which could be manually managed, and rewrite the Java code to run faster and occupy less RAM. We literally did that for some critical parts of the Java standard libraries, and left other parts as the off-the-shelf Java implementations.
> **Comments:**
> **username_9** (2023-09-26): Aside from all this complicated stuff, there's also things like the Boehm GC for C code. It uses gc "by default" because you call `GC_MALLOC` "by default" instead of `malloc`. But you can still manually manage memory by calling `malloc` and `free` instead. That's what you asked for, right? ;-)  
> **username_12** (2023-09-26): Welcome, this was really interesting to read! Given this answer and another one, perhaps embedded/mobile development is a niche where this kind of "mixing" of memory management strategies makes sense and is worth it?  
> **username_9** (2023-09-26): @kaya3: certainly I'd say that embedded is a realm where developers are more likely to want to write code that looks like systems programming, even though what you're actually writing is definitely an application. Of course, "mobile" today means a device as powerful or more than the desktop I was using in 2000 to write this code for what we then called smartphones, and now would call barely fit to use as a paperweight.  
> **username_12** (2023-09-26): I was thinking also that perhaps FFI is less of an option in (some) embedded contexts; if you can't farm out the manual memory management to a separate lower-level language then it makes more sense to have it alongside GC-managed code in the same language. But this is just my speculation since I have no experience with embedded programming.  
> **username_9** (2023-09-26): Yeah, although I think probably that model does also work very well: you "just" need the high-level language to have a reasonably lightweight interpreter. I don't do embedded any more and I don't know the state of the art, but even back then there was always a tension between the code that you want to write quickly, and the code that you want to hand-optimise. You didn't want to write everything in assembly if you could avoid it: but at the time we considered even C kind of heavyweight unless you were very careful what std libraries you called.  
> **username_9** (2023-09-26): Also, sad to say, at the time we usually got genuinely better code out of hand-written assembly than out of GCC, even from just "bashing out" the assembly the obvious way, before trying to optimise. There was just always something GCC felt obliged to keep track of that you didn't really need, like it missed a zap that should be obvious, or didn't reuse stack, or just something. I expect that has probably changed now, and you have to be really quite good at assembly to beat GCC without trying. Likely this was not entirely GCC's fault, and our GCC backend could have been better too.  
> **username_9** (2023-09-26): JNI is at least in the same neighbourhood too. Absolutely it's an FFI between Java and C. But if you choose to ignore the Java programming language, write all your code in C, and third parties send you bytecode to execute along with it; then that's the general idea. Down in your JNI code you can choose between allocating C objects to manage yourself, or allocating Java objects to be GCed. We effectively had JNI with zero change of context, because we were writing in the assembly for the same instruction set that Java bytecode got translated to when loaded. There was no Java interpreter.  
> **username_15** (2023-09-29): //I expect that has probably changed now, and you have to be really quite good at assembly to beat GCC without trying.//  That's certainly not true for the Cortex-M0.  Sometimes it seems the only way to coax clang and gcc to generate good code for a loop is to use `volatile` to prevent gcc from replacing an add-to-register instruction within a loop with a combination of load-constant-into-register instruction, move register, and add-to-register instruction (all within the loop), or prevent clang from transforming something like `while (x >= 0)` into `while (x > (stepsize-1))`.  
---
### Answer by username_15 on 2023-09-26. Score: 2
An abstraction model which uses explicit object disposal will be essentially as good or better than garbage collection in most usage cases, but at least three usage cases where garbage collection is better are biggies:

1. It's useful to be able to pass around references to immutable data-holding objects, including objects of mutable type which are wrapped by immutable objects and will never change after wrapper construction is complete, as proxies for the data contained therein, without having to know or care how many references exist to any particular object.
2. Sometimes it's useful for objects to perform actions on behalf of other objects, without having any particular interest in those objects. For example, an object might notify an "updates counter" object every time it's updated, but if the last entity that's interested in the update counter goes away, the monitored object's reference to the update counter should go away at some point(\*)
3. A tracing garbage-collector can ensure that after an object is notified its services will no longer be required, notifies other entities upon which it relies likewise, and consequently becomes useless, references to that object will continue to remain as valid references to a useless object, and the object can then deterministically reject attempts to continue using it rather than having such requests act upon other arbitrary objects, with unpredictable effects.

The existence of a garbage collector in no way obviates the need for explicit deterministic resource cleanup, but even the best imaginable mechanisms for explicit resource cleanup in no way obviate the above advantages for a tracing garbage collector.

(\*) I've yet to see a framework handle this particularly well; a design I'd like to see would be to have event handlers provide an "are you still interested?" function which would allow an event handler to indicate whether further notifications were required. If the event handler has been told its services are no longer required, it could let any object that sends it an event know (via return value) that it should be unsubscribed. To guard against memory leaks, events with event lists could send "are you still interested?" messages to two subscribers (if that many exist) every time a new subscription is added. Unused subscription entries for dead objects might remain around indefinitely, but there would be no way for subscription lists to fill up with an unbounded number of such entries, since old once would be getting cleaned out while new ones were added.
> **Comments:**
> **username_9** (2023-09-27): (2) sounds a bit like the "entity that's interested in the update counter" should hold a hard reference, and the "monitored object" should hold a weak reference. Then, when the last entity that's interested in the update counter goes away, the monitored object's reference to the update counter can no longer be acquired - this failure to acquire could be interpreted the same way you'd interpret a negative response to "are you still interested?". Of course weak references have a non-trivial implementation overhead. Am I missing something else, or is the problem that cost?  
> **username_15** (2023-09-27): @SteveJessop: It's possible to manage views using ownership and "non-owning" pointers, even without GC, but that requires adding another layer of notifications to deal with changes to subscriptions, and all interactions among notifications need to be handled in thread-safe and deadlock-free fashion.  If one has an object which is e.g. supposed to count how many times some monitored event happens, but not do anything other than hold the count, allowing multiple objects who are interested in the thing being watched to share that object without the observer having to care can simplify things.  
---
### Answer by username_19 on 2023-09-27. Score: 2
Yes

Qt, when used with C++, does this.
The Qt data structures are managed. However, nothing prevents you from using C++ or even plain old C pointers at the same time.
One could even use Qt and STL data structures at the same time. For example, QVector and std::vector. Can be useful when interfacing with libraries which don't use Qt.
---
### Answer by username_20 on 2023-09-26. Score: 1
Yes.

Object Pascal behaves like that. Class instances and ordinary blocks of memory are subject to manual allocation and deallocation, while strings and dynamic arrays are reference-counted.
---
