<p>I'm implementing my own statically-typed programming language and I'm not too happy with my own approach to types. At the moment, I'm relying on mapping a textual representation of a type to an index in a hash map, and then trying to parse that for &quot;sub types&quot; (for things like &quot;arrays <em>of</em> a type&quot;, &quot;pointers to arrays of a type&quot;, etc).</p>
<p>How do compiler implementations parse and represent types, in order to track them throughout the compilation process, compare them for equivalence as well as handle those variants or subtypes?</p>

<language-design><types><implementation><compilers><static-typing>
What are common approaches to representing types within a compiler?
CC BY-SA 4.0
2023-05-17T15:13:17.990Z
Welcome to PLDI! This seems like a pretty broad question, since there's sooo many variations in how type systems work, and so many different parts of them. Compiler design is a whole field, after all.
rydwolf
CC BY-SA 4.0
2023-05-17T15:21:42.937Z
That's fair. I got a little excited about this stack-exchange because I'm quite new to language implementation. I'd like to narrow down the question so it's useful, but I don't really know how best to.
springogeek
CC BY-SA 4.0
2023-05-17T16:26:48.423Z
Could you please be more specific? Are you asking about type systems per se (e.g. type checking algorithms)? Compiler architecture (e.g. how to thread types through IRs)? How to parse and represent types as a data structure?
James Martin
CC BY-SA 4.0
2023-05-17T16:41:56.420Z
I'm interested in all of that, really, but for this question, more the latter two. Parsing and representing types as a data structure within the compiler was my main focus though, so let's go that way
springogeek
CC BY-SA 4.0
2023-05-17T17:09:31.427Z
Thanks for editing your question. I have voted to reopen.
chrisaycock
<p>The easiest way to track type information---as well as identifier resolution---is by storing semantic information in the abstract syntax tree (AST). Some compilers refer to this semantically upgraded AST as the high-level intermediary representation (HIR).</p>
<p>As for how to compare types, the specifics depend heavily on your language's semantics. But let's say that your compiler's semantic analysis (sema) phase walks the AST to produce the HIR. Then for each step through the tree, you ask whether the user's action is permitted under your type rules. Eg., a function call is allowed because the function's type signature is met by the operands the user has provided. Any information gleamed from visiting that AST node is embedded in a new HIR node.</p>
<p>As for parsing types, treat your types the same way you would any other identifier. Your grammar should only see types the same way it sees variables; don't handle this during syntax. Let the sema phase determine the kind of identifier (type vs variable) and whether it was appropriate in the user's code.</p>
<p>Lastly, I will give a tip I learned the hard way... If you ever plan on lowering your AST/HIR to a mid-level IR (MIR) or to a virtual machine (VM) at some point in the future, just have the MIR/VM store its opcodes in the compiler's initial HIR at startup. I.e., the VM effectively creates the initial HIR so that the compiler can compare user calls against built-in functions against the VM's opcodes. Then the codgen phase only needs to emit those opcodes in addition to any resolved operands.</p>

CC BY-SA 4.0
2023-05-17T17:35:51.600Z
2023-05-17T17:35:51.600Z
chrisaycock
CC BY-SA 4.0
2023-09-29T22:08:44.433Z
@springogeek: Depending upon the language, the AST may be a rather natural representation for the type, at least if one makes a few tweaks like constant folding and having an "user identifier" node contain a link to the definition.
supercat
CC BY-SA 4.0
2023-05-17T18:02:06.643Z
This covers how type information propagates through the AST. How would you represent the type (and it's properties?) Thinking about both composite/record types as well as something like the types in ML-style languages
springogeek
<p>Storing them in a dynamic array seems like a good idea in my language.</p>
<p>I have Crystal-like union typing in my language, so types can be arbitrarily large, and I needed to solve the issue of type storage.</p>
<p>If it's a hashmap, you'd have the HashMap type that takes two arguments: the Key Type and the Value Type. In a dynamic array, <code>HashMap&lt;string, int&gt;</code> can simply look like this:
<code>[TYPE_HASHMAP, TYPE_STRING, TYPE_INT]</code></p>
<p>Then you can just probe the list to figure out the type of something.</p>
<p>A function like this:
<code>fn foo(a: int, b: string) -&gt; bool</code></p>
<p>Can look like this in a type list:
<code>[TYPE_FUNCTION, 2, TYPE_INT, TYPE_STRING, TYPE_BOOL]</code></p>
<p>If the types have subsub types, say a function like this
<code>fn foo(map: HashMap&lt;string, Array&lt;int&gt;&gt;) -&gt; void</code></p>
<p>It can look like this:
<code>[TYPE_FUNCTION, 1, TYPE_HASHMAP, TYPE_STRING, TYPE_ARRAY, TYPE_INT, TYPE_VOID]</code>
(First the function type, with 1 argument, then the HashMap type, the hash map key type, the hash map value type which is an array with its value type as an int and finally the return type)</p>
<p>This makes it easy to compare equality through linear probing. It's easy to manage too I think. It's sort of like types can take arguments.</p>

CC BY-SA 4.0
2023-05-17T19:13:24.233Z
2023-05-17T19:13:24.233Z
Lucrecious
CC BY-SA 4.0
2023-05-17T19:27:01.147Z
This is an interesting approach. I assume you aren't using an enum for types, because that would restrict your ability to create new types during compilation, right?
springogeek
CC BY-SA 4.0
2023-05-20T01:56:09.090Z
Not necessarily. An enum can be converted to an integer in most languages so custom types use the same enums but within a user range, in my case it's between 15-60000. That way I can keep the size of individual types capped at 16 bits.
Lucrecious
<p>Types, like many other things in compilers (programs), are usually represented as some encoding of a discriminated union - similar to how we usually store and manipulate ASTs.</p>
<p>For example, suppose we were implementing a type checker for STLC (Simply Typed Lambda Calculus), extended with some integral arithmetic primitives. We start by thinking about what kinds of types our language will have, so we can go about representing them. For our flavour of STLC, we only really need a ground type for integer literals (e.g., <code>4 : int</code>), some notion of type variables (unknowns, <code>x : 't0</code>), and arrow types (e.g., <code>f : 't0 -&gt; int</code>).</p>
<p>In OCaml (or any other language with algebraic datatypes), we could do this very naturally:</p>
<pre><code>type typ =
  | Tint
  | Tvar of int
  | Tarrow of typ * typ
</code></pre>
<p>In languages without ADTs, such as Java or C++, the common choice is to encode discriminated unions (tagged unions) as a class hierarchy, down-casting when necessary. For example, in Java we may have:</p>
<pre><code>public abstract class Type {
  public class Int extends Type {}
  public class Var extends Type {
    private int name;
    public Var(int name) {
      this.name = name;
    }
    // ...
  }
  public class Arrow extends Type {
    private Type domain, codomain;
    public Arrow (Type domain, Type codomain) {
      this.domain = domain;
      this.codomain = codomain;
    }
    // ...
  }

}
</code></pre>
<p>In order to work with these encodings, we have pattern matching in OCaml and various methods in Java (<code>instanceof</code>, using a visitor design pattern, the modern <code>switch</code>, etc.).</p>
<pre><code>let foo = function
  | Tarrow (l, r) -&gt; (* ... *)
  | _ -&gt; (* ...  *)
</code></pre>
<pre><code>public void foo(Type ty) {
  if (ty instanceof Type.Arrow arrow) {
    // use arrow in some way
  }
}
</code></pre>
<p>The reason why people tend to enjoy writing compilers in OCaml, Haskell, Standard ML, etc. is that the encoding of recursive data (inductively defined) is incredibly convenient and the means by which to work with it (structural recursion) is very clean to express with pattern matching.</p>
<p>The troubles of dealing with recursive type abbreviations, type definitions that are not well founded, type equality, shadowed type definitions, etc. is usually up to book keeping. Of course, structural equality is not so difficult to implement, but many languages have a more strict notion of what it means for two types to be equal (within the type system).</p>
<hr />
<p>Another thing to look out for is that type representations are quite often implemented in such a way to support a common operation performed on them in the context of type inference; destructive unification. So, their definitions are often augmented with constructors specifically for inducing a union-find forest (union-find is an ideal implementation strategy for the usual kinds of first-order unification we see in Hindley-Milner type systems).</p>
<p>If we look at the very approachable Caml Light codebase, we see its representation of types isn't too far from what I've described: <a href="https://github.com/camllight/camllight/blob/0cc537de0846393322058dbb26449427bfc76786/windows/src/compiler/globals.ml#L27-L48" rel="noreferrer">Caml Light: compiler/globals.ml</a>. We see that type variables (<code>Tvar</code>s) hold a mutable reference cell that's mutated during destructive unification. It's typical that these union-find artefacts are removed after inference in a process sometimes called &quot;zonking&quot; (from the GHC codebase - effectively, replacing all annotated types with their representative).</p>
<p>Similarly, if we look at Clang, we'll see its <a href="https://clang.llvm.org/doxygen/classclang_1_1PointerType.html" rel="noreferrer">clang::PointerType</a> is a subtype of <code>clang::Type</code> and appears to store the type being pointed to, with a relevant accessor: <code>getPointeeType()</code> - we can imagine that the idiomatic encoding in an ML would be something like:</p>
<pre><code>type typ =
  | (* ... *)
  | Pointer of typ
</code></pre>
<p>So, really: like many things in this area of programming, it tends to boil down to how your implementation language allows you to encode and work with one of the most important ideas in programming: discriminated unions. It is incredibly unfortunate that many mainstream languages neglected this concern until recently.</p>
<p>I hope this helps, it should be clear that many type formation rules can be encoded as inductive data structures in a fairly straightforward way. As for the other problems, those are largely bookkeeping (see the concept of &quot;stamps&quot; in the Caml Light and OCaml compilers).</p>

CC BY-SA 4.0
2023-05-17T23:11:04.600Z
2023-05-17T23:11:04.600Z
Colin James
CC BY-SA 4.0
2023-05-18T04:23:05.910Z
Wow, this is a very comprehensive answer, thank you. I'm not quite sure about a few terms you use, though. "Destructive unification" and "union-find forest". Could you clarify those for me?
springogeek
CC BY-SA 4.0
2023-05-18T06:41:21.750Z
Many type inference algorithms rely on unification for working out of two types are "compatible" (made equal up to a unifying substitution). Old unification algorithms such as that of Robinson's are incredibly inefficient for real world compilers (they reify substitutions as actual things to compose and apply), and so most make use of destructive unification (based on the union-find data structure, as a forest, encoded directly in the type representation). This is effectively the difference between algorithm J and algorithm W - J has a magic "unify" function that's taken to be done this way.
Colin James
CC BY-SA 4.0
2023-05-18T06:43:45.653Z
I recommend introductory resources such as Oleg's telling of the core of the Caml Light/OCaml typechecker: https://okmij.org/ftp/ML/generalization/sound_eager.ml - the effective idea is that when you're dealing with polymorphism, where some constraint specialises a type variable, you can do the substitution implicitly as part of the union-find data structure. When we unify `'a` with `int`, we really rewrite the cell representing the unbound type variable `'a` to be a link to the representation of the type `int`. 

I recommend getting familiar with union-find, it's key to many fun algorithms.
Colin James
CC BY-SA 4.0
2023-05-18T12:22:31.917Z
I appreciate the resource link. I have no experience with OCaml/Caml at all, so reading those sources is a bit challenging.
springogeek
CC BY-SA 4.0
2023-05-20T22:02:51.387Z
This is an excellent answer, except for one point. This answer discusses rather simple type systems, such as Caml Light's (which is pretty much just Hindley-Milner). A lot of what you dismiss as “bookkeeping” is very much nontrivial. With features such as structural subtyping, modules and recursive types, the OCaml type checker is considerably more complex. With many advanced features, you can't have a canonical representation of types (or don't want to, because it would be unprintable in error messages).
Gilles 'SO- stop being evil'
# What are common approaches to representing types within a compiler?
Posted by: username_0 on 2023-05-17
I'm implementing my own statically-typed programming language and I'm not too happy with my own approach to types. At the moment, I'm relying on mapping a textual representation of a type to an index in a hash map, and then trying to parse that for "sub types" (for things like "arrays *of* a type", "pointers to arrays of a type", etc).

How do compiler implementations parse and represent types, in order to track them throughout the compilation process, compare them for equivalence as well as handle those variants or subtypes?
**Tags:** language-design types implementation compilers static-typing
> **Comments:**
> **username_1**: Welcome to PLDI! This seems like a pretty broad question, since there's sooo many variations in how type systems work, and so many different parts of them. Compiler design is a whole field, after all.  
> **username_0**: That's fair. I got a little excited about this stack-exchange because I'm quite new to language implementation. I'd like to narrow down the question so it's useful, but I don't really know how best to.  
> **username_2**: Could you please be more specific? Are you asking about type systems per se (e.g. type checking algorithms)? Compiler architecture (e.g. how to thread types through IRs)? How to parse and represent types as a data structure?  
> **username_0**: I'm interested in all of that, really, but for this question, more the latter two. Parsing and representing types as a data structure within the compiler was my main focus though, so let's go that way  
> **username_3**: Thanks for editing your question. I have voted to reopen.  
---
## 3 answers
### Answer by username_6 on 2023-05-17. Score: 5 | Accepted answer
Types, like many other things in compilers (programs), are usually represented as some encoding of a discriminated union - similar to how we usually store and manipulate ASTs.
For example, suppose we were implementing a type checker for STLC (Simply Typed Lambda Calculus), extended with some integral arithmetic primitives. We start by thinking about what kinds of types our language will have, so we can go about representing them. For our flavour of STLC, we only really need a ground type for integer literals (e.g., 4 : int), some notion of type variables (unknowns, x : 't0), and arrow types (e.g., f : 't0 -> int).
In OCaml (or any other language with algebraic datatypes), we could do this very naturally:
type typ =
  | Tint
  | Tvar of int
  | Tarrow of typ * typ

In languages without ADTs, such as Java or C++, the common choice is to encode discriminated unions (tagged unions) as a class hierarchy, down-casting when necessary. For example, in Java we may have:
public abstract class Type {
  public class Int extends Type {}
  public class Var extends Type {
    private int name;
    public Var(int name) {
      this.name = name;
    }
    // ...
  }
  public class Arrow extends Type {
    private Type domain, codomain;
    public Arrow (Type domain, Type codomain) {
      this.domain = domain;
      this.codomain = codomain;
    }
    // ...
  }

}

In order to work with these encodings, we have pattern matching in OCaml and various methods in Java (instanceof, using a visitor design pattern, the modern switch, etc.).
let foo = function
  | Tarrow (l, r) -> (* ... *)
  | _ -> (* ...  *)

public void foo(Type ty) {
  if (ty instanceof Type.Arrow arrow) {
    // use arrow in some way
  }
}

The reason why people tend to enjoy writing compilers in OCaml, Haskell, Standard ML, etc. is that the encoding of recursive data (inductively defined) is incredibly convenient and the means by which to work with it (structural recursion) is very clean to express with pattern matching.
The troubles of dealing with recursive type abbreviations, type definitions that are not well founded, type equality, shadowed type definitions, etc. is usually up to book keeping. Of course, structural equality is not so difficult to implement, but many languages have a more strict notion of what it means for two types to be equal (within the type system).

Another thing to look out for is that type representations are quite often implemented in such a way to support a common operation performed on them in the context of type inference; destructive unification. So, their definitions are often augmented with constructors specifically for inducing a union-find forest (union-find is an ideal implementation strategy for the usual kinds of first-order unification we see in Hindley-Milner type systems).
If we look at the very approachable Caml Light codebase, we see its representation of types isn't too far from what I've described: Caml Light: compiler/globals.ml. We see that type variables (Tvars) hold a mutable reference cell that's mutated during destructive unification. It's typical that these union-find artefacts are removed after inference in a process sometimes called "zonking" (from the GHC codebase - effectively, replacing all annotated types with their representative).
Similarly, if we look at Clang, we'll see its clang::PointerType is a subtype of clang::Type and appears to store the type being pointed to, with a relevant accessor: getPointeeType() - we can imagine that the idiomatic encoding in an ML would be something like:
type typ =
  | (* ... *)
  | Pointer of typ

So, really: like many things in this area of programming, it tends to boil down to how your implementation language allows you to encode and work with one of the most important ideas in programming: discriminated unions. It is incredibly unfortunate that many mainstream languages neglected this concern until recently.
I hope this helps, it should be clear that many type formation rules can be encoded as inductive data structures in a fairly straightforward way. As for the other problems, those are largely bookkeeping (see the concept of "stamps" in the Caml Light and OCaml compilers).
> **Comments:**
> **username_0** (2023-05-18): Wow, this is a very comprehensive answer, thank you. I'm not quite sure about a few terms you use, though. "Destructive unification" and "union-find forest". Could you clarify those for me?  
> **username_6** (2023-05-18): Many type inference algorithms rely on unification for working out of two types are "compatible" (made equal up to a unifying substitution). Old unification algorithms such as that of Robinson's are incredibly inefficient for real world compilers (they reify substitutions as actual things to compose and apply), and so most make use of destructive unification (based on the union-find data structure, as a forest, encoded directly in the type representation). This is effectively the difference between algorithm J and algorithm W - J has a magic "unify" function that's taken to be done this way.  
> **username_6** (2023-05-18): I recommend introductory resources such as Oleg's telling of the core of the Caml Light/OCaml typechecker: https://okmij.org/ftp/ML/generalization/sound_eager.ml - the effective idea is that when you're dealing with polymorphism, where some constraint specialises a type variable, you can do the substitution implicitly as part of the union-find data structure. When we unify `'a` with `int`, we really rewrite the cell representing the unbound type variable `'a` to be a link to the representation of the type `int`. 

I recommend getting familiar with union-find, it's key to many fun algorithms.  
> **username_0** (2023-05-18): I appreciate the resource link. I have no experience with OCaml/Caml at all, so reading those sources is a bit challenging.  
> **username_7** (2023-05-20): This is an excellent answer, except for one point. This answer discusses rather simple type systems, such as Caml Light's (which is pretty much just Hindley-Milner). A lot of what you dismiss as “bookkeeping” is very much nontrivial. With features such as structural subtyping, modules and recursive types, the OCaml type checker is considerably more complex. With many advanced features, you can't have a canonical representation of types (or don't want to, because it would be unprintable in error messages).  
---
### Answer by username_3 on 2023-05-17. Score: 2
The easiest way to track type information---as well as identifier resolution---is by storing semantic information in the abstract syntax tree (AST). Some compilers refer to this semantically upgraded AST as the high-level intermediary representation (HIR).

As for how to compare types, the specifics depend heavily on your language's semantics. But let's say that your compiler's semantic analysis (sema) phase walks the AST to produce the HIR. Then for each step through the tree, you ask whether the user's action is permitted under your type rules. Eg., a function call is allowed because the function's type signature is met by the operands the user has provided. Any information gleamed from visiting that AST node is embedded in a new HIR node.

As for parsing types, treat your types the same way you would any other identifier. Your grammar should only see types the same way it sees variables; don't handle this during syntax. Let the sema phase determine the kind of identifier (type vs variable) and whether it was appropriate in the user's code.

Lastly, I will give a tip I learned the hard way... If you ever plan on lowering your AST/HIR to a mid-level IR (MIR) or to a virtual machine (VM) at some point in the future, just have the MIR/VM store its opcodes in the compiler's initial HIR at startup. I.e., the VM effectively creates the initial HIR so that the compiler can compare user calls against built-in functions against the VM's opcodes. Then the codgen phase only needs to emit those opcodes in addition to any resolved operands.
> **Comments:**
> **username_0** (2023-05-17): This covers how type information propagates through the AST. How would you represent the type (and it's properties?) Thinking about both composite/record types as well as something like the types in ML-style languages  
> **username_4** (2023-09-29): @springogeek: Depending upon the language, the AST may be a rather natural representation for the type, at least if one makes a few tweaks like constant folding and having an "user identifier" node contain a link to the definition.  
---
### Answer by username_5 on 2023-05-17. Score: 2
Storing them in a dynamic array seems like a good idea in my language.

I have Crystal-like union typing in my language, so types can be arbitrarily large, and I needed to solve the issue of type storage.

If it's a hashmap, you'd have the HashMap type that takes two arguments: the Key Type and the Value Type. In a dynamic array, `HashMap` can simply look like this:
`[TYPE_HASHMAP, TYPE_STRING, TYPE_INT]`

Then you can just probe the list to figure out the type of something.

A function like this:
`fn foo(a: int, b: string) -> bool`

Can look like this in a type list:
`[TYPE_FUNCTION, 2, TYPE_INT, TYPE_STRING, TYPE_BOOL]`

If the types have subsub types, say a function like this
`fn foo(map: HashMap>) -> void`

It can look like this:
`[TYPE_FUNCTION, 1, TYPE_HASHMAP, TYPE_STRING, TYPE_ARRAY, TYPE_INT, TYPE_VOID]`
(First the function type, with 1 argument, then the HashMap type, the hash map key type, the hash map value type which is an array with its value type as an int and finally the return type)

This makes it easy to compare equality through linear probing. It's easy to manage too I think. It's sort of like types can take arguments.
> **Comments:**
> **username_0** (2023-05-17): This is an interesting approach. I assume you aren't using an enum for types, because that would restrict your ability to create new types during compilation, right?  
> **username_5** (2023-05-20): Not necessarily. An enum can be converted to an integer in most languages so custom types use the same enums but within a user range, in my case it's between 15-60000. That way I can keep the size of individual types capped at 16 bits.  
---
