<p>No language that I know of directly has a mechanism for embedding GPU shaders in the code. Third party libraries such as OpenGL are required, and they usually have the shader source code as just a string, and is compiled at runtime, making the system much less efficient.</p>
<p>I would expect some languages out there to have native support for GPU execution, so that the compiler can identify shaders and compile time accordingly, instead of compiling them at runtime.</p>
<p>The only language that half-does this is Metal shading language, which is loaded by the Metal library on the Swift side post-compilation. But that is still not an intrinsic capability of the Swift language.</p>
<p>What would supporting GPU execution in a language take and why is this never seen?</p>

<shading>
How could a language support GPU execution cleanly?
CC BY-SA 4.0
2023-07-03T18:32:13.920Z
Is Metal really a Swift thing? I was under the impression (and I could be wrong) that Metal is an Apple Objective-C thing, and Swift can use it because Swift can use Apple Objective-C.
Bbrk24
CC BY-SA 4.0
2023-07-03T18:35:01.470Z
@Bbrk24 It's a Swift & Objective-C thing. There are some discrepancies, such as in swift there is a method are called `makeCommandBuffer` and in Objective-C `newCommandBuffer`.
CPlus
CC BY-SA 4.0
2023-07-03T18:36:41.663Z
That makes sense. I wonder if the Objective-C one is just annotated `NS_SWIFT_NAME("makeCommandBuffer")` or some equivalent, but I suppose that's neither here nor there.
Bbrk24
CC BY-SA 4.0
2023-07-04T08:42:55.353Z
Julia does it in quite a transparent way, Mojo is apparently going to offload code implicitly to GPUs.
SK-logic
CC BY-SA 4.0
2023-07-04T09:52:09.403Z
To my knowledge, the reason shaders are compiled on the fly, is that the shader will be used on a variety of different GPUs. The only alternative would be to provide compiled shaders for each GPU out there, which seems more complex, and not sure how possible it is.
Tom Tsagkatos
CC BY-SA 4.0
2023-07-04T10:14:39.117Z
@TomTsagk there is at least SPIR-V and PTX, so you can compile your kernels into an intermediate representation which is portable across GPUs and is still usable on a CPU as  well.
SK-logic
<p>It is sometimes seen ─ shaders written in HLSL (the shader language used by DirectX) <a href="https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-part1" rel="nofollow noreferrer">can also be precompiled</a>. But I think in practice this is usually only done when targeting games consoles, where you know in advance exactly what graphics hardware the shaders will run on. Optimising for the specific hardware will have a bigger impact than the short amount of time it takes on initial load to compile a shader from source.</p>
<p>There is a possible middle-ground, which is compiling shaders to a bytecode language which can then be compiled again at runtime by a JIT for the specific graphics hardware. The bytecode language would have to be standardised so that it could be supported by device drivers which contain optimising compilers for the specific hardware.</p>
<p>In practice, such a bytecode language might not exist for the graphics API you're using*, and I doubt that it would be much faster than compiling from source code as a string. Shader languages are pretty simple compared to general-purpose languages, and the optimisations that can be done platform-independently (at your compile-time) are probably very cheap compared to a lot of other things that would still need to be done at runtime. Bytecode would also take up less space, but the size of shader source code is negligible compared to other assets anyway.</p>
<p><sup>* There are intermediate bytecode languages for DirectX and Vulkan, but I don't think there is one for OpenGL.</sup></p>
<hr />
<p>All of this explains why shaders are typically distributed as source code. That said, it would still be possible for a compiler to allow shaders to be written in the same language as normal code: you would just need to be able to transpile from that language to the shader language. But this feature would not be as useful as it sounds.</p>
<p>It's not <em>quite</em> true that shaders are typically distributed as source code ─ in most large applications, the source code for a shader is not just compiled at runtime, it is <em>created</em> at runtime, either by string concatenation or something more advanced. This is done because applications want to detect the capabilities of the hardware, and offer the user the option to disable certain features for performance reasons, so there might be thousands or millions of combinations of which parts of a shader need to be used, and the correct combination will only be known at runtime.</p>
<p>So it's not enough to just be able to transpile your language to a shader language ─ you need to transpile it to fragments (no pun intended) of shader language source code, to be concatenated at runtime based on the appropriate conditions.</p>

CC BY-SA 4.0
2023-07-03T18:42:32.773Z
2023-07-06T11:24:41.847Z
2023-07-06T11:24:41.847Z
kaya3
<p>Most programming languages can not be cross-compiled to shader code because the structures and capabilities of GPU code are just too different from those of CPU code.</p>
<p>However, embedding shader code would be an option. Embedding one programming language in another is hardly uncharted territory. For example, C and C++ have the <code>asm</code> keyword for ages.</p>
<p>Why does no programming language do that for shader code, though?</p>
<ul>
<li>There are just too many shader languages to choose from. GLSL, HLSL, PSSL, MSL, WGSL... So either you choose exactly one to support (greatly narrowing the applicability of your language), try to support as many as possible (which is a ton of work) or you invent yet another shading language representing the smallest common denominator between them all (which will only support the most basic functionality).</li>
<li>CPU and GPU generally communicate &quot;at arms length&quot; with each other through a graphics API like OpenGL, DirectX, Vulkan or Metal. Abstracting that away is often not desired or useful.</li>
<li>Pre-compiling shaders is already possible. But it is often not desired, because they are indeed created at runtime based on the configuration of the system they run on.</li>
</ul>

CC BY-SA 4.0
2023-07-04T10:43:27.587Z
2023-07-04T10:43:27.587Z
Philipp
<blockquote>
<p>How could a language support GPU execution cleanly?</p>
</blockquote>
<p>There are several aspects or interpretations to this question, so maybe the question should be split into narrower ones...</p>
<p><strong>First</strong>, there's a question &quot;GPUs (and similar hardware) have particular execution and memory model that's rather different from common CPUs — how can a programming language support it?&quot;</p>
<p>There are quite a number of languages — both standalone and (embedded) DSLs — aimed specifically at (GP)GPU execution: various shader languages (as you've mentioned), CUDA, OpenCL, <a href="https://halide-lang.org/" rel="nofollow noreferrer">Halide</a>, <a href="https://ispc.github.io/" rel="nofollow noreferrer">ISPC</a>, etc.</p>
<p>Though in my opinion, the <em>cleanest</em> GPGPU language is <a href="https://futhark-lang.org" rel="nofollow noreferrer">Futhark</a>. From my view it fully leverages non-Turing-completeness of GPU computations and provides a sweet spot of high-level features for automatic parallelism:</p>
<ul>
<li>arrays-oriented total functional language</li>
<li>limited easy to use form of dependent types for static dimensionality checking</li>
<li>limited form of linear types for locally mutable computations</li>
<li>some other quality-of-life things</li>
</ul>
<p><strong>Second</strong>, there's a question &quot;how can we <em>embed</em> computation offloading to GPU (and other accelerators) in a general-purpose language?&quot;</p>
<p>I haven't seen a really <em>clean</em> and neat solution in this space, though Julia GPU ecosystem supports a huge chunk of the language completely seamlessly (including GC support for GPU). There are a number of Python libraries/extensions supporting various subsets of the language and providing automated GPU/TPU/etc offloading.</p>
<p><a href="https://anydsl.github.io" rel="nofollow noreferrer">AnyDSL</a> presents arguably one of the cleanest approaches through seamless integration of several DSLs on top of the common highly optimized intermediate representation, but that's a very much research project.</p>
<p><strong>Third</strong>, there's a question &quot;how can we support <em>graphics programming</em> in a general-purpose language?&quot; Preferably in a <em>safe</em> fashion.</p>
<p>I don't have much knowledge and experience on this front, but from what I've heard you can't precompile shaders to actual machine code because GPUs use completely different proprietary ISAs. The only common API is some common standardized API — OpenGL/Vulcan + shaders/SPIR-V. Thus you can't statically check anything beyond the API/shader model either.</p>
<p>That being said, from practical standpoint many Rust game developers care a lot about statically checking as much as possible, thus there are several projects integrating shaders into a graphics framework through procedural macros and checking statically whatever they can.</p>
<p>Also WebGPU project explicitly builds a safe API, at least in a sense you can't corrupt GPU memory or somehow &quot;brick&quot; it or anything. And they strive to check as much as possible statically and as little as necessary dynamically.</p>
<p>I hope this overview will help clarifying and narrowing down follow-up questions...</p>

CC BY-SA 4.0
2023-07-04T09:55:51.807Z
2023-07-04T09:55:51.807Z
Alex Chichigin
# How could a language support GPU execution cleanly?
Posted by: username_0 on 2023-07-03
No language that I know of directly has a mechanism for embedding GPU shaders in the code. Third party libraries such as OpenGL are required, and they usually have the shader source code as just a string, and is compiled at runtime, making the system much less efficient.

I would expect some languages out there to have native support for GPU execution, so that the compiler can identify shaders and compile time accordingly, instead of compiling them at runtime.

The only language that half-does this is Metal shading language, which is loaded by the Metal library on the Swift side post-compilation. But that is still not an intrinsic capability of the Swift language.

What would supporting GPU execution in a language take and why is this never seen?
**Tags:** shading
> **Comments:**
> **username_1**: Is Metal really a Swift thing? I was under the impression (and I could be wrong) that Metal is an Apple Objective-C thing, and Swift can use it because Swift can use Apple Objective-C.  
> **username_0**: @Bbrk24 It's a Swift & Objective-C thing. There are some discrepancies, such as in swift there is a method are called `makeCommandBuffer` and in Objective-C `newCommandBuffer`.  
> **username_1**: That makes sense. I wonder if the Objective-C one is just annotated `NS_SWIFT_NAME("makeCommandBuffer")` or some equivalent, but I suppose that's neither here nor there.  
> **username_2**: Julia does it in quite a transparent way, Mojo is apparently going to offload code implicitly to GPUs.  
> **username_3**: To my knowledge, the reason shaders are compiled on the fly, is that the shader will be used on a variety of different GPUs. The only alternative would be to provide compiled shaders for each GPU out there, which seems more complex, and not sure how possible it is.  
> **username_2**: @TomTsagk there is at least SPIR-V and PTX, so you can compile your kernels into an intermediate representation which is portable across GPUs and is still usable on a CPU as  well.  
---
## 3 answers
### Answer by username_4 on 2023-07-03. Score: 6
It is sometimes seen ─ shaders written in HLSL (the shader language used by DirectX) [can also be precompiled](https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-part1). But I think in practice this is usually only done when targeting games consoles, where you know in advance exactly what graphics hardware the shaders will run on. Optimising for the specific hardware will have a bigger impact than the short amount of time it takes on initial load to compile a shader from source.

There is a possible middle-ground, which is compiling shaders to a bytecode language which can then be compiled again at runtime by a JIT for the specific graphics hardware. The bytecode language would have to be standardised so that it could be supported by device drivers which contain optimising compilers for the specific hardware.

In practice, such a bytecode language might not exist for the graphics API you're using\*, and I doubt that it would be much faster than compiling from source code as a string. Shader languages are pretty simple compared to general-purpose languages, and the optimisations that can be done platform-independently (at your compile-time) are probably very cheap compared to a lot of other things that would still need to be done at runtime. Bytecode would also take up less space, but the size of shader source code is negligible compared to other assets anyway.

\* There are intermediate bytecode languages for DirectX and Vulkan, but I don't think there is one for OpenGL.

---

All of this explains why shaders are typically distributed as source code. That said, it would still be possible for a compiler to allow shaders to be written in the same language as normal code: you would just need to be able to transpile from that language to the shader language. But this feature would not be as useful as it sounds.

It's not *quite* true that shaders are typically distributed as source code ─ in most large applications, the source code for a shader is not just compiled at runtime, it is *created* at runtime, either by string concatenation or something more advanced. This is done because applications want to detect the capabilities of the hardware, and offer the user the option to disable certain features for performance reasons, so there might be thousands or millions of combinations of which parts of a shader need to be used, and the correct combination will only be known at runtime.

So it's not enough to just be able to transpile your language to a shader language ─ you need to transpile it to fragments (no pun intended) of shader language source code, to be concatenated at runtime based on the appropriate conditions.
---
### Answer by username_6 on 2023-07-04. Score: 3
> How could a language support GPU execution cleanly?

There are several aspects or interpretations to this question, so maybe the question should be split into narrower ones...

**First**, there's a question "GPUs (and similar hardware) have particular execution and memory model that's rather different from common CPUs — how can a programming language support it?"

There are quite a number of languages — both standalone and (embedded) DSLs — aimed specifically at (GP)GPU execution: various shader languages (as you've mentioned), CUDA, OpenCL, [Halide](https://halide-lang.org/), [ISPC](https://ispc.github.io/), etc.

Though in my opinion, the *cleanest* GPGPU language is [Futhark](https://futhark-lang.org). From my view it fully leverages non-Turing-completeness of GPU computations and provides a sweet spot of high-level features for automatic parallelism:

* arrays-oriented total functional language
* limited easy to use form of dependent types for static dimensionality checking
* limited form of linear types for locally mutable computations
* some other quality-of-life things

**Second**, there's a question "how can we *embed* computation offloading to GPU (and other accelerators) in a general-purpose language?"

I haven't seen a really *clean* and neat solution in this space, though Julia GPU ecosystem supports a huge chunk of the language completely seamlessly (including GC support for GPU). There are a number of Python libraries/extensions supporting various subsets of the language and providing automated GPU/TPU/etc offloading.

[AnyDSL](https://anydsl.github.io) presents arguably one of the cleanest approaches through seamless integration of several DSLs on top of the common highly optimized intermediate representation, but that's a very much research project.

**Third**, there's a question "how can we support *graphics programming* in a general-purpose language?" Preferably in a *safe* fashion.

I don't have much knowledge and experience on this front, but from what I've heard you can't precompile shaders to actual machine code because GPUs use completely different proprietary ISAs. The only common API is some common standardized API — OpenGL/Vulcan + shaders/SPIR-V. Thus you can't statically check anything beyond the API/shader model either.

That being said, from practical standpoint many Rust game developers care a lot about statically checking as much as possible, thus there are several projects integrating shaders into a graphics framework through procedural macros and checking statically whatever they can.

Also WebGPU project explicitly builds a safe API, at least in a sense you can't corrupt GPU memory or somehow "brick" it or anything. And they strive to check as much as possible statically and as little as necessary dynamically.

I hope this overview will help clarifying and narrowing down follow-up questions...
---
### Answer by username_5 on 2023-07-04. Score: 1
Most programming languages can not be cross-compiled to shader code because the structures and capabilities of GPU code are just too different from those of CPU code.

However, embedding shader code would be an option. Embedding one programming language in another is hardly uncharted territory. For example, C and C++ have the `asm` keyword for ages.

Why does no programming language do that for shader code, though?

* There are just too many shader languages to choose from. GLSL, HLSL, PSSL, MSL, WGSL... So either you choose exactly one to support (greatly narrowing the applicability of your language), try to support as many as possible (which is a ton of work) or you invent yet another shading language representing the smallest common denominator between them all (which will only support the most basic functionality).
* CPU and GPU generally communicate "at arms length" with each other through a graphics API like OpenGL, DirectX, Vulkan or Metal. Abstracting that away is often not desired or useful.
* Pre-compiling shaders is already possible. But it is often not desired, because they are indeed created at runtime based on the configuration of the system they run on.
---
