<p>Java has the <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Override.html" rel="nofollow noreferrer"><code>@Override</code></a> annotation. This annotation, when applied on a method, basically says that this method is intended to be an override of a superclass method. If the annotation is applied on a method that does not share signatures with any method in the superclass, the Java compiler gives an error.</p>
<p>When Kotlin came along, <code>Override</code> was promoted from an annotation to the keyword <a href="https://kotlinlang.org/docs/inheritance.html#overriding-methods" rel="nofollow noreferrer"><code>override</code></a>.<sup>1</sup> Conversely, Java's <code>volatile</code> keyword, which signified an uncached variable<sup>2</sup>, has its counterpart in the <a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.jvm/-volatile/" rel="nofollow noreferrer"><code>@Volatile</code></a> annotation in Kotlin. The same situation applies to <code>synchronized</code>, <code>transient</code>, and I think a few more. Additionally, <code>synchronized</code> blocks (not methods) aren't even a core part of Kotlin; they are a function taking a lambda (which looks very much like block syntax in Kotlin).</p>
<p>Another notable example is Python. when dealing with class-based programming, a lot of Python's features are accessible through annotations<sup>3</sup>. For example, static methods are annotated with <a href="https://docs.python.org/3/library/functions.html#classmethod" rel="nofollow noreferrer"><code>@classmethod</code></a><sup>4</sup>, whereas Java uses the <code>static</code> keyword and Kotlin uses a <code>companion object</code>, both syntactic features. A similar situation occurs with <a href="https://docs.python.org/3/library/dataclasses.html" rel="nofollow noreferrer"><code>@dataclass</code></a>, with the Kotlin and Java equivalents being <code>data class</code> and <code>record</code>, respectively. To get abstract classes and enums in Python, you have to extend a class found in the standard library, while they are both core syntactic features in Java and Kotlin.</p>
<p>This kind of game where you either put something as a core syntax feature or a part of the standard library intrigues me. I imagine Rust could have made a <code>deriving</code> keyword or use the colon syntax (as it already does with traits) instead of the <code>derive</code> macro (which is similar enough to annotations). What considerations must be taken into account when determining what features go into the standard library versus what makes it as a dedicated syntactic feature?</p>
<hr />
<p><sup>1</sup><sub>As a matter of personal preference, I prefer the annotation.</sub></p>
<p><sup>2</sup><sub>As far as I can tell.</sub></p>
<p><sup>3</sup><sub>Actually called decorators, but I'll call 'em annotations for the sake of not confusing people.</sub></p>
<p><sup>4</sup><sub>Yes, I know about <code>@staticmethod</code>, but <code>@classmethod</code> is closer to the Java/Kotlin equivalents</sub></p>

<syntax><standard-library><keywords>
How do language designers determine what feature flags are part of the standard library and what are part of syntax?
<p>Override is a good example. Depending on how your language works override might be required as a keyword, because the compiler does something before annotations can be processed. Volatile is also a good example, because it exists traditionally and stems from a time where annotations did not exist. But processing of volatile happens very late, so it can be turned into an attribute today. Popular languages that have attributes today often did not have them when their first versions were published.</p>

CC BY-SA 4.0
2023-08-19T08:27:15.903Z
2023-08-19T08:27:15.903Z
feldentm
<p>In Java, the <code>@Override</code> annotation was introduced when the language had already been in use for a while, so they had to maintain backwards compatibility (i.e. keep old code working without it). This meant that they both couldn't afford to introduce a new keyword (which could collide with previously valid identifiers), and had to make the override specification optional, so an annotation was pretty much a forced choice. In Python, the philosophy is to implement as much as possible in the form of library features, not language ones, so instead of making static and class methods a special case, they've introduced the concept of descriptor objects, and have implemented <code>staticmethod</code>, <code>classmethod</code> and <code>property</code> in the standard library based on it.</p>
<p>In general, I'd say these represent the two most common motivations affecting this choice.</p>
<ul>
<li>An established language cannot really afford to introduce new keywords <sub>(yes, soft keywords are another potential solution)</sub>, so they are often forced to fall back to attributes</li>
<li>If the language aims to minimize the core builtin part, it makes sense to only provide keywords for features that cannot be implemented otherwise</li>
<li>On the other hand, if there's a feature that could be implemented in the standard library, but is deemed common enough to deserve a more convenient syntax, a keyword could be introduced for syntactic sugar. (Like what Python 3.5 did with <code>async</code> and <code>await</code> instead of relying on specific generators)</li>
</ul>

CC BY-SA 4.0
2023-06-22T21:25:19.487Z
2023-06-22T21:25:19.487Z
abel1502
<p>I'll answer from my perspective as a language designer.</p>
<p><strong>tl;dr</strong>: it is impossible to generalize design decisions.</p>
<h2>Generalizing Design Decisions</h2>
<p>In the mind of every language designer, and indeed, designers of every <em>thing</em> that has ever existed, is a set of constraints they have to design by.</p>
<p>The best designers make these constraints explicit in something like design documents, but often, these constraints are implicit, only existing in the mind of the designer.</p>
<p>Even then, they might somehow be brought up to justify decisions after the fact (see the <a href="https://en.wikipedia.org/wiki/Choice-supportive_bias" rel="noreferrer">choice-supportive bias</a>, aka <strong>rationalization</strong>), but again, the best designers consider constraints <em>before</em> making a decision, not <em>after</em>.</p>
<p>@abel1502 has an excellent answer laying out the concrete constraints that those specific languages had, whether before or after. Unfortunately, the answer does only address concrete cases. This is because generalizing design decisions can only take place in the context of understanding the design process and design factors.</p>
<p>So to answer why some design is a certain way, you can only answer the question for <em>that design</em>.</p>
<p>But once you know the design process and enumerate the factors, you can figure out the answers for other designs more easily.</p>
<h2>Concrete Examples</h2>
<p>With that said, I will talk about a few concrete examples.</p>
<p>C and C++ have added a few keywords since C89 and whatever the equivalent C++ standard is. However, they <em>also</em> needed backwards compatibility, and they don't really have annotations, per se.</p>
<p>Fortunately, the designers thought forward and gave themselves an out: user code is not permitted to use identifiers that begin with an underscore (<code>_</code>).</p>
<p>So the new keywords all invariably begin with an underscore (<code>_Generic</code>, <code>_Static_assert</code>, etc.)</p>
<p>So C and C++ <em>do</em> have annotation-like things now, by adding specific keywords. One example is the <code>_Noreturn</code> keyword.</p>
<p>But wait, C and C++ have macros, and they added a <code>static_assert</code> macro, one does <em>not</em> begin with an underscore.</p>
<p>The reason this is still &quot;backwards compatible&quot; is because of several reasons:</p>
<ul>
<li>You have to <code>#include</code> (import) the <code>assert.h</code> header to get that macro.</li>
<li>You will only get that macro if your toolchain is for the newer versions of C. If your toolchain is for an old version, the version you probably wrote your code for, you won't get that macro.</li>
</ul>
<p>So in essence, C had a reserved set of keywords that still maintain backwards compatibility, along with the equivalent of Rust editions and requiring you to ask for the new stuff by import.</p>
<p>My language has the same sort of thing. You cannot have two or more consecutive underscores in an identifier (for name mangling), and keywords are associated with <em>packages</em> not built-in to the language.</p>
<p>Sure, if I were to add keywords to the base standard library, that would break code because the standard library is essentially imported the same as the Python <code>from std import *</code>.</p>
<p>But I can always add keywords to subpackages instead, which you would have to directly import and then use the package name as a prefix, meaning that you would have to <em>choose</em> to do so.</p>
<p>In other words, if you make the user do work to get your new stuff, that is sufficient to maintain backwards compatibility.</p>
<p>Another example is Zig where &quot;annotations&quot; are not keywords nor annotations, but really built-in functions and mostly correspond to LLVM-specific extensions. These came about because of implementation concerns.</p>
<h2>Design Factors for This Case</h2>
<p>So now we have a partial list of design factors relating to annotations versus keywords:</p>
<ul>
<li>Backwards compatibility.</li>
<li>Language versions.</li>
<li>Making the user do work to get the new stuff.</li>
<li>Implementations.</li>
</ul>
<p>Use this list to start judging this type of design decision, and I think you'll easily identify most or all of the constraints that mattered to the language designer.</p>

CC BY-SA 4.0
2023-06-24T14:51:58.220Z
2023-06-28T03:15:06.387Z
2023-06-28T03:15:06.387Z
Gavin D. Howard
CC BY-SA 4.0
2023-08-22T14:22:47.983Z
Nitpick: C++ handled it similarly to but differently from C. There are no [C++ keywords](https://en.cppreference.com/w/cpp/keyword) starting with underscore, and `static_assert` is a keyword, not a macro. Also, C++ does have [attributes](https://en.cppreference.com/w/cpp/language/attributes).
Pablo H
CC BY-SA 4.0
2024-07-22T00:38:38.857Z
Small point of order: C doesn't reserve _all_ identifiers beginning with an underscore; only those starting with an underscore and then either an upper-case letter or another underscore. That's to say, matching the regex `^_[_A-Z]`. Identifiers like `_abc` and `_100` are available to the programmer.
Martin Kealey
# How do language designers determine what feature flags are part of the standard library and what are part of syntax?
Posted by: username_0 on 2023-06-21
Java has the [`@Override`](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Override.html) annotation. This annotation, when applied on a method, basically says that this method is intended to be an override of a superclass method. If the annotation is applied on a method that does not share signatures with any method in the superclass, the Java compiler gives an error.

When Kotlin came along, `Override` was promoted from an annotation to the keyword [`override`](https://kotlinlang.org/docs/inheritance.html#overriding-methods).1 Conversely, Java's `volatile` keyword, which signified an uncached variable2, has its counterpart in the [`@Volatile`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.jvm/-volatile/) annotation in Kotlin. The same situation applies to `synchronized`, `transient`, and I think a few more. Additionally, `synchronized` blocks (not methods) aren't even a core part of Kotlin; they are a function taking a lambda (which looks very much like block syntax in Kotlin).

Another notable example is Python. when dealing with class-based programming, a lot of Python's features are accessible through annotations3. For example, static methods are annotated with [`@classmethod`](https://docs.python.org/3/library/functions.html#classmethod)4, whereas Java uses the `static` keyword and Kotlin uses a `companion object`, both syntactic features. A similar situation occurs with [`@dataclass`](https://docs.python.org/3/library/dataclasses.html), with the Kotlin and Java equivalents being `data class` and `record`, respectively. To get abstract classes and enums in Python, you have to extend a class found in the standard library, while they are both core syntactic features in Java and Kotlin.

This kind of game where you either put something as a core syntax feature or a part of the standard library intrigues me. I imagine Rust could have made a `deriving` keyword or use the colon syntax (as it already does with traits) instead of the `derive` macro (which is similar enough to annotations). What considerations must be taken into account when determining what features go into the standard library versus what makes it as a dedicated syntactic feature?

---

1As a matter of personal preference, I prefer the annotation.

2As far as I can tell.

3Actually called decorators, but I'll call 'em annotations for the sake of not confusing people.

4Yes, I know about `@staticmethod`, but `@classmethod` is closer to the Java/Kotlin equivalents
**Tags:** syntax standard-library keywords
---
## 3 answers
### Answer by username_2 on 2023-06-22. Score: 11
In Java, the `@Override` annotation was introduced when the language had already been in use for a while, so they had to maintain backwards compatibility (i.e. keep old code working without it). This meant that they both couldn't afford to introduce a new keyword (which could collide with previously valid identifiers), and had to make the override specification optional, so an annotation was pretty much a forced choice. In Python, the philosophy is to implement as much as possible in the form of library features, not language ones, so instead of making static and class methods a special case, they've introduced the concept of descriptor objects, and have implemented `staticmethod`, `classmethod` and `property` in the standard library based on it.

In general, I'd say these represent the two most common motivations affecting this choice.

* An established language cannot really afford to introduce new keywords (yes, soft keywords are another potential solution), so they are often forced to fall back to attributes
* If the language aims to minimize the core builtin part, it makes sense to only provide keywords for features that cannot be implemented otherwise
* On the other hand, if there's a feature that could be implemented in the standard library, but is deemed common enough to deserve a more convenient syntax, a keyword could be introduced for syntactic sugar. (Like what Python 3.5 did with `async` and `await` instead of relying on specific generators)
---
### Answer by username_3 on 2023-06-24. Score: 10
I'll answer from my perspective as a language designer.

**tl;dr**: it is impossible to generalize design decisions.

## Generalizing Design Decisions

In the mind of every language designer, and indeed, designers of every *thing* that has ever existed, is a set of constraints they have to design by.

The best designers make these constraints explicit in something like design documents, but often, these constraints are implicit, only existing in the mind of the designer.

Even then, they might somehow be brought up to justify decisions after the fact (see the [choice-supportive bias](https://en.wikipedia.org/wiki/Choice-supportive_bias), aka **rationalization**), but again, the best designers consider constraints *before* making a decision, not *after*.

@abel1502 has an excellent answer laying out the concrete constraints that those specific languages had, whether before or after. Unfortunately, the answer does only address concrete cases. This is because generalizing design decisions can only take place in the context of understanding the design process and design factors.

So to answer why some design is a certain way, you can only answer the question for *that design*.

But once you know the design process and enumerate the factors, you can figure out the answers for other designs more easily.

## Concrete Examples

With that said, I will talk about a few concrete examples.

C and C++ have added a few keywords since C89 and whatever the equivalent C++ standard is. However, they *also* needed backwards compatibility, and they don't really have annotations, per se.

Fortunately, the designers thought forward and gave themselves an out: user code is not permitted to use identifiers that begin with an underscore (`_`).

So the new keywords all invariably begin with an underscore (`_Generic`, `_Static_assert`, etc.)

So C and C++ *do* have annotation-like things now, by adding specific keywords. One example is the `_Noreturn` keyword.

But wait, C and C++ have macros, and they added a `static_assert` macro, one does *not* begin with an underscore.

The reason this is still "backwards compatible" is because of several reasons:

* You have to `#include` (import) the `assert.h` header to get that macro.
* You will only get that macro if your toolchain is for the newer versions of C. If your toolchain is for an old version, the version you probably wrote your code for, you won't get that macro.

So in essence, C had a reserved set of keywords that still maintain backwards compatibility, along with the equivalent of Rust editions and requiring you to ask for the new stuff by import.

My language has the same sort of thing. You cannot have two or more consecutive underscores in an identifier (for name mangling), and keywords are associated with *packages* not built-in to the language.

Sure, if I were to add keywords to the base standard library, that would break code because the standard library is essentially imported the same as the Python `from std import *`.

But I can always add keywords to subpackages instead, which you would have to directly import and then use the package name as a prefix, meaning that you would have to *choose* to do so.

In other words, if you make the user do work to get your new stuff, that is sufficient to maintain backwards compatibility.

Another example is Zig where "annotations" are not keywords nor annotations, but really built-in functions and mostly correspond to LLVM-specific extensions. These came about because of implementation concerns.

## Design Factors for This Case

So now we have a partial list of design factors relating to annotations versus keywords:

* Backwards compatibility.
* Language versions.
* Making the user do work to get the new stuff.
* Implementations.

Use this list to start judging this type of design decision, and I think you'll easily identify most or all of the constraints that mattered to the language designer.
> **Comments:**
> **username_4** (2023-08-22): Nitpick: C++ handled it similarly to but differently from C. There are no [C++ keywords](https://en.cppreference.com/w/cpp/keyword) starting with underscore, and `static_assert` is a keyword, not a macro. Also, C++ does have [attributes](https://en.cppreference.com/w/cpp/language/attributes).  
> **username_5** (2024-07-22): Small point of order: C doesn't reserve _all_ identifiers beginning with an underscore; only those starting with an underscore and then either an upper-case letter or another underscore. That's to say, matching the regex `^_[_A-Z]`. Identifiers like `_abc` and `_100` are available to the programmer.  
---
### Answer by username_1 on 2023-08-19. Score: 1
Override is a good example. Depending on how your language works override might be required as a keyword, because the compiler does something before annotations can be processed. Volatile is also a good example, because it exists traditionally and stems from a time where annotations did not exist. But processing of volatile happens very late, so it can be turned into an attribute today. Popular languages that have attributes today often did not have them when their first versions were published.
---
