<p>Important prefacing notes:</p>
<ul>
<li>When I say &quot;language&quot; in this question, I'm not referring to a &quot;language&quot; in the &quot;more holistic sense&quot; that includes things like its standard library and tooling. Those can have their own question posts. Here, I'm referring to things like the language's <strong>syntax, grammar, and semantics</strong>.</li>
<li>This question is about features for anticipating backwards-incompatible changes such as <a href="https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1881r0.html" rel="nofollow noreferrer">the C++ epochs proposal</a>, Rust editions, and Perl's <code>use v5.8</code>. The question is not about human-related processes of doing proposals / RFC-like things.</li>
</ul>
<p>I've touched on this in <a href="https://languagedesign.stackexchange.com/a/889/251">my answer</a> to <a href="https://languagedesign.stackexchange.com/q/888/251">How can isolated focus on syntax be counterproductive in the earlier stages of a language&#39;s design?</a>, where in part of the answer, I state:</p>
<blockquote>
<p>Rarely does a language exist through time without its designers or users wanting it to change, and/or actually trying to change it. [...] The choices you make now [...] can interfere with choices that you may want to make in the future. It is probably impossible to prevent every such kind of unhappy scenario, but there are techniques to design a language for evolution and/or extension.</p>
</blockquote>
<hr />
<p>For the purposes of this question, I'm not interested in answers based purely on hypothetical reasoning absent of the context of any existing language. This is a case-study-oriented question. I want to see either one of the following:</p>
<ul>
<li>How past attempts to build evolvability into a language work (in terms of mechanics), why those designers chose that method, and how that has fared in hindsight.</li>
<li>What options designers of a specific language are exploring or considering to &quot;add evolvability&quot; to their language, why they are considering those approaches, and how they evaluate the effectiveness of those approaches (such as what advantages and disadvantages they see or expect to see).</li>
</ul>
<hr />
<p>For the option / route of past attempts, when you answer &quot;how has this technique fared&quot;, I want something <em>concrete</em>. To do that, for example, you can:</p>
<ul>
<li>Speak to real pain points that you have observed users of the language repeatedly express about the evolution mechanism (please link to examples)</li>
<li>Refer to &quot;design regrets&quot; that designers of that language have identified, with reference to the reasoning behind those regrets (please link to the information source)</li>
</ul>
<p>It would be appreciated if you try to uncover any ties in this to how the language itself generally works and/or what its goals are (I.e. give caveats that prevent misintrepreting the learnings as a result of separating information from important context).</p>
<p>If the answer to &quot;how has this technique fared&quot; is &quot;it's too early to tell&quot;, that's fine, but please say so, explain why, and try to remember to come back and edit your answer if that information emerges.</p>
<hr />
<p>Please either optimize your answer post for depth or for breadth. For example, you can pick one language and post about it in depth, or you can try to give a &quot;big picture view&quot; for many languages (I'd say <em>at least</em> three if you want to go this route) and focus on <em>comparing them</em>.</p>
<p><sup>Once more answers start trickling in, I may or may not try experimenting with putting a table of contents in this question post of the answers, ordered by some objective-criteria (Ex. alphabetical. not sure what to do yet for comparison/breadth-oriented answers) (<a href="https://languagedesign.meta.stackexchange.com/a/180/251">related on meta</a>)</sup></p>

<case-studies><evolution>
What techniques have existing languages used or considered using to enable their evolution? Why were they chosen/considered? How have they fared?
CC BY-SA 4.0
2023-05-22T07:44:22.993Z
I'd also point out that having three question marks in just the title is an indication that a question isn't focused enough.
kaya3
CC BY-SA 4.0
2023-05-22T09:09:32.020Z
My reading of this question, which might not be right, is that it's asking about language-level features for anticipating backwards-incompatible changes (so Perl's `use v5.8` might be an example), rather than change management (so Java's JEP would not be an example). Is that accurate?
Michael Homer
CC BY-SA 4.0
2023-05-22T09:12:53.903Z
@MichaelHomer yes that's correct. I'm not asking about things like proposal / RFC-like processes here. Suggestions to clarify / disambiguate the question are welcome. As someone who doesn't know much about the technical details of this part of PL design (only knows that it exists from hearing it talked about sometimes in CppCon videos), perhaps it's showing that it's difficult for me to find how to communicate effectively what I'm looking for here.
starball
CC BY-SA 4.0
2023-05-22T09:34:33.337Z
In that case I've misunderstood the question. I now realise you're talking about language features like Python's `from __future__ import ...`, which should definitely be on-topic. I retract my close-vote and my downvote.
kaya3
CC BY-SA 4.0
2023-05-22T09:47:30.600Z
I think I could propose an edit but it would radically change the way the question is asked ─ I'd have the title be something like *"When languages are designed to allow for future breaking changes, how does this impact users?"* ─ and give more examples in the question (so Python's `__future__`, Rust editions, Javascript's `"use strict"` might fit here too).
kaya3
CC BY-SA 4.0
2023-05-22T09:52:48.443Z
@kaya3 thanks for the suggestion, but that's not my intent for this question. I want this question to be about "case-studies", and not necessarily limit the discussion of considerations to talking about effects on users (although I assume that's usually a primary consideration). Your suggestion could potentially be a separate question post.
starball
CC BY-SA 4.0
2023-05-22T10:11:52.947Z
I think it would be better if you phrased your question purely in terms of what information you are seeking, not what format you want that information to be presented in ("case study" or otherwise). Paragraphs like *"For example, you can pick one language and post about it in depth, or you can try to give a "big picture view" for many languages (I'd say at least three if you want to go this route) and focus on comparing them"* make sense if you are setting homework for your students, but not in a Stack Exchange Q&A.
kaya3
<h2>Rust edition system</h2>
<p>The idea behind the edition system is that the compiler will always keep supporting old syntax, but each project can choose which version of the syntax to use, while still being able to use libraries and the ecosystem that might use a different edition for it's syntax.</p>
<p><a href="https://doc.rust-lang.org/edition-guide/editions/index.html" rel="nofollow noreferrer">Quote from the Rust Book</a></p>
<blockquote>
<p>Editions are the mechanism we use to solve this problem. When we want to release a feature that would otherwise be backwards incompatible, we do so as part of a new Rust edition. Editions are opt-in, and so existing crates do not see these changes until they explicitly migrate over to the new edition. This means that even the latest version of Rust will still not treat async as a keyword, unless edition 2018 or later is chosen. This choice is made per crate as part of its Cargo.toml. New crates created by cargo new are always configured to use the latest stable edition.</p>
</blockquote>
<p>Rust does also include tools that allow automated upgrading to a newer edition.</p>
<h3>Why this was chosen</h3>
<p>The main reason cited for picking this technique is:</p>
<ul>
<li><a href="https://blog.rust-lang.org/2021/05/11/edition-2021.html" rel="nofollow noreferrer">Does not split the ecosystem</a>, unlike just releasing a new major version</li>
</ul>
<p>Some of my own thoughts about the implications of picking such a system. Likely the rust team considered these too but I don't have official documentation for it:</p>
<ul>
<li>Requires maintaining all old syntax perpetually</li>
<li>Is specified at a package level, not at a file level like <code>#[feature]</code> or pythons' <code>from __future__ import</code>. This improves consistency in a package but when moving files between projects can cause issues</li>
<li>Requires the new edition code to still be compatible with old code, so radical changes to things like memory layout or calling conventions are off the table</li>
<li>One &quot;edition&quot; can include many features at once, there is no option to mix and match specific syntax features unlike <code>#[feature]</code>  and <code>from __future__ import</code>. Improves consistency but makes it harder to upgrade to newer editions since multiple things need to be fixed at once.</li>
<li>Very much a personal opinion, but I find it weird to use years to mark editions. A semver-like system would be more familiar to most programmers. Whenever I see my edition is 2021 in the year 2023 it feels old. I know rust also has a semver-style versioning system it uses for a different reason but if you are building a language with no such constraint I'd suggest just counting up from 1 instead of using years, unless you are committing to a new major version every year. This bullet point is just my personal opinion though.</li>
</ul>
<h3>How this has fared</h3>
<p>There have been 2 editions in rust so far (2018 and 2021). They have been used for things like:</p>
<ul>
<li>Removing deprecated syntax and features</li>
<li>Reserving syntax for future use</li>
<li>Improving consistency</li>
</ul>
<p>In general the editions seem mostly to be changes that are just one step of a more complex change, most of which would go via the standard experimental/nightly/stable cycle, rather than containing new features itself.</p>
<p>In the community this system is well received, and I can confirm old packages still work which is nice. There is some criticism on the specific exclusion of type inference rules from the stability guarantees but overall this seems to rarely be an issue.</p>
<h4>The <code>IntoIter</code> issue</h4>
<p>One downside that was discovered is that editions provide no way to make breaking changes in library code, since it is shared. This has been a problem with implementing <code>IntoIter</code> for arrays:</p>
<blockquote>
<p>It has been suggested many times to &quot;only implement IntoIterator for arrays in Rust 2021&quot;. However, this is simply not possible. You can't have a trait implementation exist in one edition and not in another, since editions can be mixed.</p>
</blockquote>
<p><a href="https://blog.rust-lang.org/2021/05/11/edition-2021.html#intoiterator-for-arrays" rel="nofollow noreferrer">From the blog post on edition 2021</a></p>
<p>To &quot;fix&quot; this the rust team added a special case to parsing in old editions to special case the <code>into_iter</code> function on arrays to not automatically de-reference. If a more traditional breaking change approach had been used such workarounds would not have been necessary.</p>

CC BY-SA 4.0
2023-05-22T08:52:44.053Z
2023-05-23T07:20:05.343Z
2023-05-23T07:20:05.343Z
mousetail 'he-him'
# What techniques have existing languages used or considered using to enable their evolution? Why were they chosen/considered? How have they fared?
Posted by: username_0 on 2023-05-21
Important prefacing notes:

* When I say "language" in this question, I'm not referring to a "language" in the "more holistic sense" that includes things like its standard library and tooling. Those can have their own question posts. Here, I'm referring to things like the language's **syntax, grammar, and semantics**.
* This question is about features for anticipating backwards-incompatible changes such as [the C++ epochs proposal](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1881r0.html), Rust editions, and Perl's `use v5.8`. The question is not about human-related processes of doing proposals / RFC-like things.

I've touched on this in [my answer](https://languagedesign.stackexchange.com/a/889/251) to [How can isolated focus on syntax be counterproductive in the earlier stages of a language's design?](https://languagedesign.stackexchange.com/q/888/251), where in part of the answer, I state:

> Rarely does a language exist through time without its designers or users wanting it to change, and/or actually trying to change it. [...] The choices you make now [...] can interfere with choices that you may want to make in the future. It is probably impossible to prevent every such kind of unhappy scenario, but there are techniques to design a language for evolution and/or extension.

---

For the purposes of this question, I'm not interested in answers based purely on hypothetical reasoning absent of the context of any existing language. This is a case-study-oriented question. I want to see either one of the following:

* How past attempts to build evolvability into a language work (in terms of mechanics), why those designers chose that method, and how that has fared in hindsight.
* What options designers of a specific language are exploring or considering to "add evolvability" to their language, why they are considering those approaches, and how they evaluate the effectiveness of those approaches (such as what advantages and disadvantages they see or expect to see).

---

For the option / route of past attempts, when you answer "how has this technique fared", I want something *concrete*. To do that, for example, you can:

* Speak to real pain points that you have observed users of the language repeatedly express about the evolution mechanism (please link to examples)
* Refer to "design regrets" that designers of that language have identified, with reference to the reasoning behind those regrets (please link to the information source)

It would be appreciated if you try to uncover any ties in this to how the language itself generally works and/or what its goals are (I.e. give caveats that prevent misintrepreting the learnings as a result of separating information from important context).

If the answer to "how has this technique fared" is "it's too early to tell", that's fine, but please say so, explain why, and try to remember to come back and edit your answer if that information emerges.

---

Please either optimize your answer post for depth or for breadth. For example, you can pick one language and post about it in depth, or you can try to give a "big picture view" for many languages (I'd say *at least* three if you want to go this route) and focus on *comparing them*.

Once more answers start trickling in, I may or may not try experimenting with putting a table of contents in this question post of the answers, ordered by some objective-criteria (Ex. alphabetical. not sure what to do yet for comparison/breadth-oriented answers) ([related on meta](https://languagedesign.meta.stackexchange.com/a/180/251))
**Tags:** case-studies evolution
> **Comments:**
> **username_1**: I'd also point out that having three question marks in just the title is an indication that a question isn't focused enough.  
> **username_2**: My reading of this question, which might not be right, is that it's asking about language-level features for anticipating backwards-incompatible changes (so Perl's `use v5.8` might be an example), rather than change management (so Java's JEP would not be an example). Is that accurate?  
> **username_0**: @MichaelHomer yes that's correct. I'm not asking about things like proposal / RFC-like processes here. Suggestions to clarify / disambiguate the question are welcome. As someone who doesn't know much about the technical details of this part of PL design (only knows that it exists from hearing it talked about sometimes in CppCon videos), perhaps it's showing that it's difficult for me to find how to communicate effectively what I'm looking for here.  
> **username_1**: In that case I've misunderstood the question. I now realise you're talking about language features like Python's `from __future__ import ...`, which should definitely be on-topic. I retract my close-vote and my downvote.  
> **username_1**: I think I could propose an edit but it would radically change the way the question is asked ─ I'd have the title be something like *"When languages are designed to allow for future breaking changes, how does this impact users?"* ─ and give more examples in the question (so Python's `__future__`, Rust editions, Javascript's `"use strict"` might fit here too).  
> **username_0**: @kaya3 thanks for the suggestion, but that's not my intent for this question. I want this question to be about "case-studies", and not necessarily limit the discussion of considerations to talking about effects on users (although I assume that's usually a primary consideration). Your suggestion could potentially be a separate question post.  
> **username_1**: I think it would be better if you phrased your question purely in terms of what information you are seeking, not what format you want that information to be presented in ("case study" or otherwise). Paragraphs like *"For example, you can pick one language and post about it in depth, or you can try to give a "big picture view" for many languages (I'd say at least three if you want to go this route) and focus on comparing them"* make sense if you are setting homework for your students, but not in a Stack Exchange Q&A.  
---
## 1 answers
### Answer by username_3 on 2023-05-22. Score: 8
## Rust edition system

The idea behind the edition system is that the compiler will always keep supporting old syntax, but each project can choose which version of the syntax to use, while still being able to use libraries and the ecosystem that might use a different edition for it's syntax.

[Quote from the Rust Book](https://doc.rust-lang.org/edition-guide/editions/index.html)

> Editions are the mechanism we use to solve this problem. When we want to release a feature that would otherwise be backwards incompatible, we do so as part of a new Rust edition. Editions are opt-in, and so existing crates do not see these changes until they explicitly migrate over to the new edition. This means that even the latest version of Rust will still not treat async as a keyword, unless edition 2018 or later is chosen. This choice is made per crate as part of its Cargo.toml. New crates created by cargo new are always configured to use the latest stable edition.

Rust does also include tools that allow automated upgrading to a newer edition.

### Why this was chosen

The main reason cited for picking this technique is:

* [Does not split the ecosystem](https://blog.rust-lang.org/2021/05/11/edition-2021.html), unlike just releasing a new major version

Some of my own thoughts about the implications of picking such a system. Likely the rust team considered these too but I don't have official documentation for it:

* Requires maintaining all old syntax perpetually
* Is specified at a package level, not at a file level like `#[feature]` or pythons' `from __future__ import`. This improves consistency in a package but when moving files between projects can cause issues
* Requires the new edition code to still be compatible with old code, so radical changes to things like memory layout or calling conventions are off the table
* One "edition" can include many features at once, there is no option to mix and match specific syntax features unlike `#[feature]` and `from __future__ import`. Improves consistency but makes it harder to upgrade to newer editions since multiple things need to be fixed at once.
* Very much a personal opinion, but I find it weird to use years to mark editions. A semver-like system would be more familiar to most programmers. Whenever I see my edition is 2021 in the year 2023 it feels old. I know rust also has a semver-style versioning system it uses for a different reason but if you are building a language with no such constraint I'd suggest just counting up from 1 instead of using years, unless you are committing to a new major version every year. This bullet point is just my personal opinion though.

### How this has fared

There have been 2 editions in rust so far (2018 and 2021). They have been used for things like:

* Removing deprecated syntax and features
* Reserving syntax for future use
* Improving consistency

In general the editions seem mostly to be changes that are just one step of a more complex change, most of which would go via the standard experimental/nightly/stable cycle, rather than containing new features itself.

In the community this system is well received, and I can confirm old packages still work which is nice. There is some criticism on the specific exclusion of type inference rules from the stability guarantees but overall this seems to rarely be an issue.

#### The `IntoIter` issue

One downside that was discovered is that editions provide no way to make breaking changes in library code, since it is shared. This has been a problem with implementing `IntoIter` for arrays:

> It has been suggested many times to "only implement IntoIterator for arrays in Rust 2021". However, this is simply not possible. You can't have a trait implementation exist in one edition and not in another, since editions can be mixed.

[From the blog post on edition 2021](https://blog.rust-lang.org/2021/05/11/edition-2021.html#intoiterator-for-arrays)

To "fix" this the rust team added a special case to parsing in old editions to special case the `into_iter` function on arrays to not automatically de-reference. If a more traditional breaking change approach had been used such workarounds would not have been necessary.
---
