<p>Okay, so this might be a silly question. I'm a big fan of functional programming paradigms in higher-level scripting languages, so I'm probably using Rust wrong. However, I'm new to Rust and I'm curious why this isn't a thing. If someone could explain why this isn't a language feature (or provide a solid technical explanation for why it's a bad idea), I'd appreciate it!</p>
<p>So, I'm working my way through the <a href="https://rust-book.cs.brown.edu/ch03-05-control-flow.html" rel="noreferrer">Rust Book</a>. So far, I'm on board with the &quot;evaluating a block as an expression given its last line is an expression business.&quot; It seems kind of LISP-y or Swift-y to me. Anyway, Rust lets you do things like this...</p>
<pre class="lang-rust prettyprint-override"><code>let x = {
    let tmp: u32 = 1;
    tmp + 1
};

println!(&quot;The value of x is {x}&quot;);
// the value is 2
</code></pre>
<p>You can also loop over an array, providing a block where you can perform an operation with each element...</p>
<pre class="lang-rust prettyprint-override"><code>let my_arr = [1, 2, 3, 4, 5];

for item in my_arr {
    println!(&quot;The value of item is {item}&quot;);
}
// the value of item is 1-5
</code></pre>
<p>So... given the block expression rules above (coupled with Rust's seeming incredible power to infer strictly typed values), why is it that we can't use <code>for x in y</code> as a way to produce a mapped array of values? I know I'm not that strong with compiled languages, but shouldn't we be able to do something like so?</p>
<pre class="lang-rust prettyprint-override"><code>let my_arr = [1, 2, 3, 4, 5];

let new_arr = for item in my_arr { item + 1 };

for item in new_arr {
    println!(&quot;The value of item is {item}&quot;);
}
// ???
</code></pre>
<p>... which, in my little Rust head-canon here, should result in a new array of values <code>[2, 3, 4, 5, 6]</code>. It's my assumption that, if we can infer the value of variable <code>x</code> in the first example, we should be able to infer the member values of the array <code>new_arr</code> and provision for it, accordingly. Is there a reason why this <em>wouldn't</em> work in Rust? Is it contrary to some design principle of the language that I've yet to discover?</p>

<rust><foreach-loop>
Why does Rust choose not to provide `for` comprehensions?
CC BY-SA 4.0
2023-08-31T15:35:58.197Z
The reason it doesn't work is because, to use something in a for loop it specifically needs to implement `IntoIter` which consumes self. Some methods implement an `iter()` method that only borrows `self` and allows you to iterate over it multiple times but there is no trait for that and for loops will never use it unless you call it explicitly.
mousetail 'he-him'
CC BY-SA 4.0
2023-08-31T15:36:57.813Z
@mousetail Remember not to answer with comments!
rydwolf
CC BY-SA 4.0
2023-08-31T15:37:55.850Z
@RydwolfPrograms It doesn't really answer the question, the question is if it could be done in a different way, not why it currently works this way
mousetail 'he-him'
CC BY-SA 4.0
2023-08-31T15:48:05.767Z
I don't understand why the two other things (block expressions and `for` loops) would suggest that the third thing (list comprehensions) should work. The value of a `for` loop is `()`, i.e. the unit value, so `let new_arr = for ...` will declare a variabel `new_arr` with the value `()`.
kaya3
CC BY-SA 4.0
2023-08-31T15:52:24.470Z
Note that what you're calling a "closure" in the question is actually called a "block". Closures in rust have a different meaning.
Seggan
CC BY-SA 4.0
2023-08-31T15:58:59.083Z
A block expression isn't a closure in any sense, this seems to be a misconception.
kaya3
CC BY-SA 4.0
2023-08-31T15:59:22.843Z
You can use `.iter().map(|x|x+1)` to perform the effect you want
mousetail 'he-him'
<p><em><strong>Note:</strong> this answer mainly addresses the question of why <code>for</code> loops are not list comprehensions in Rust. The question title was edited later to be about having list comprehensions at all.</em></p>
<hr />
<h2>Why don't <code>for</code> loops evaluate to arrays?</h2>
<p>The feature you're looking for is called <a href="https://en.wikipedia.org/wiki/List_comprehension" rel="nofollow noreferrer">list comprehensions</a>. For example, Python has them:</p>
<pre class="lang-python prettyprint-override"><code>new_arr = [x + 1 for x in my_arr]
</code></pre>
<p>Note that even in languages which have this feature, it is not the same as a <code>for</code> loop:</p>
<pre class="lang-python prettyprint-override"><code># does nothing
for x in my_arr: x + 1

# syntax error
new_arr = for x in my_arr: x + 1
</code></pre>
<p>This makes sense because a <code>for</code> loop is a statement written for its side-effects, whereas a list comprehension is an expression used to generate a list, and the individual terms in the list comprehension are generally recommended to have no side-effects. So these are two different language constructs for two different purposes and there is no need to unify them.</p>
<p>The fact that Rust is an expression-based language (i.e. <code>for</code> loops and blocks are expressions which produce values) doesn't change this; the vast majority of the time you use a <code>for</code> loop, it's not to create a new mapped array, so it would be strange (and possibly wasteful) if this was the behaviour of all <code>for</code> loops.</p>
<p>It would also be problematic for a statically typed language to make every <code>for</code> loop a list comprehension. For example, if the <code>for</code> loop's type was an array or some other sequence, then the two branches of the conditional expression like this one would not share a type:</p>
<pre class="lang-rust prettyprint-override"><code>if xs.is_empty() {
    // type is ()
    println!(&quot;No elements&quot;);
} else {
    // type is an array of ()
    for x in xs.iter() {
        println!(&quot;{x}&quot;);
    }
}
</code></pre>
<p>So it's better in the much more common case, for the type of <code>for</code> expressions to be <code>()</code>. If you want a mapped array then you can still achieve that by different means.</p>
<hr />
<h2>What about something other than <code>for</code> loops?</h2>
<p>So then, why doesn't Rust have a <em>different</em> language construct for list comprehensions? Well, you can already use iterators, <a href="https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.map" rel="nofollow noreferrer"><code>.map()</code></a> and <a href="https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.collect" rel="nofollow noreferrer"><code>.collect()</code></a> to build a mapped data structure of any type implementing the <a href="https://doc.rust-lang.org/std/iter/trait.FromIterator.html#tymethod.from_iter" rel="nofollow noreferrer"><code>FromIterator</code></a> trait, so the only thing that list comprehensions would add would be syntax sugar for those standard library features.</p>
<p>Such syntax sugar might be valuable, but Rust may choose not to have it because of the language's philosophy that the standard library should not be privileged over third-party libraries any more than necessary; and because syntax sugar can be added by third-party code using macros (e.g. <a href="https://docs.rs/mapcomp/latest/mapcomp/" rel="nofollow noreferrer">this crate</a>) anyway if some users want it.</p>

CC BY-SA 4.0
2023-08-31T15:57:19.413Z
2023-09-01T15:17:00.550Z
2023-09-01T15:17:00.550Z
kaya3
CC BY-SA 4.0
2023-09-01T13:21:18.627Z
**Comments have been [moved to chat](https://chat.stackexchange.com/rooms/148242/discussion-on-answer-by-kaya3-why-does-rust-choose-not-to-provide-for-comprehe); please do not continue the discussion here.** Before posting a comment below this one, please review the [purposes of comments](/help/privileges/comment). Comments that do not request clarification or suggest improvements usually belong as an [answer](/help/how-to-answer), on [meta], or in [chat]. Comments continuing discussion may be removed.
lyxal
<p>Maybe because I am not a functional language guy, I don't see at all your proposed syntax as obvious. But if you were to submit an RFC proposal with such thing, the first question that comes to my mind is: what should be the type returned by <code>for</code>?</p>
<p>Neither of the obvious choice seems good:</p>
<ol>
<li>it can't return an array, because arrays have fixed size, and <code>for</code> iterates over a type implementing the trait <code>Iterator</code>, but a generic <code>Iterator</code> doesn't know its size at compile time. Nor at runtime. Heck, it doesn't even have to terminate, it is perfectly valid for an iterator to be infinite (turning a <code>for</code> into an infinite loop). Thus, array is out of question.</li>
<li>it could return a <code>Vec</code>: if the compiler detects the <code>for</code> return value is being used, it creates a <code>Vec</code> and inserts the output of every iteration. But <code>Vec</code> is a high level library class, it doesn't bode well for a fundamental language construct like <code>for</code> to create and return a <code>Vec</code>. But what is worse is that <code>Vec</code> is not even available for every rust program, because it allocates memory. If you mark your program as <code>#[no_std]</code> (which is used for bare metal programming and other limited resources environments), it won't have a default global allocator, and <code>Vec</code> and anything that allocates memory won't be available.</li>
<li>it could return an inferred <code>FromIterator</code> type, like <code>Iterator::collect()</code> does. Then you could use any container you like, including some that doesn't need to allocate for <code>#[no_std]</code> environments. To avoid allocation, each <code>for</code> iteration would have to be executed on demand from the <code>FromIterator::from_iter()</code> function. I think this completely subverts the distinction between <code>for</code> and an iterator (<code>for</code> has to be effectively turned into an iterator for this to work), and violates the principle of least surprise. We expect a <code>for</code> to be greedy and execute right now, while we expect an iterator to be lazy and only produce the next element when called, if ever. In this case, each <code>for</code> iteration would have to be called from the <code>FromIterator</code> implementation, and executed in between whatever code is in there, including the possibility of side effects, because this can be an user supplied function. It can even avoid running the iterator entirely, and then your <code>for</code> simply does not execute.</li>
</ol>
<p>Given the return type difficulty, the non-obviousness of the construct (for me at least, but I believe I am representative of the average rust programmer), and most importantly, the readily available alternatives of <code>my_arr.map(...)</code> or even <code>any_iterable.iter().map(...).collect()</code> (which is basically solution 3), I doubt this was ever seriously considered.</p>

CC BY-SA 4.0
2023-09-01T14:54:08.487Z
2023-09-01T14:59:47.303Z
2023-09-01T14:59:47.303Z
lvella
<h1>Manual memory management is complicated</h1>
<p>Rust is a <a href="https://en.wikipedia.org/wiki/Systems_programming" rel="noreferrer">systems programming</a> language. While it is very much <em>possible</em> to use Rust for general-purpose application development, it is primarily specialized to the task of low-level systems programming. The most obvious example of this is its use of manual memory management.</p>
<p>Since Rust aims to achieve full <a href="https://en.wikipedia.org/wiki/Memory_safety" rel="noreferrer">memory safety</a> within its safe subset, much of the language is built around structured mechanisms for controlling memory allocation and lifetimes. In many languages, these concerns are largely handled by the programming language, which makes it both possible and easy to, say, materialize a value from nothing and not worry about where it’s being allocated, who owns it, and how long it lives. The answers to those questions are both simple and uniform: the value is allocated in the heap, the runtime owns its memory, and it lives until it is no longer reachable. Rust does not have such luxuries, and this does not just mean that things are less automatic, it also means they are more <em>nuanced</em>.</p>
<p>In Rust, one can allocate memory in numerous different ways, and values’ lifetimes can be controlled by any of several different mechanisms. There are many different gradations of ownership, each with their own implications and tradeoffs. This is not a shortcoming of the language, it’s the <em>point</em>: people doing systems programming care about the distinctions between the potential options. The purpose of Rust is to give such programmers low-level control over these mechanisms while retaining safety, and in exchange, the programmer must be exquisitely explicit about each of the choices they make.</p>
<p><a href="https://en.wikipedia.org/wiki/List_comprehension" rel="noreferrer">List comprehensions</a> are not <em>fundamentally</em> incompatible with this attitude, but they are also far from straightforward. Here are a few examples of questions they raise:</p>
<ul>
<li><p>Where does the memory for the list returned by a comprehension come from? How long does it live for? What happens when it is freed?</p>
</li>
<li><p>If the programmer is mapping over an existing structure, is the structure’s memory reused, consuming it in the process? If so, how can the language ensure that memory is not inadvertently accessed from within the body of the loop while the structure is somehow in an invalid state?</p>
</li>
<li><p>Since values can escape the body of a list comprehension, how is the memory for local variables within the loop body managed? Can the space used by the loop be linear in the number of iterations? Do values need to be copied at the end of each iteration?</p>
</li>
<li><p>In Rust, iteration is overloadable via the <a href="https://doc.rust-lang.org/std/iter/trait.Iterator.html" rel="noreferrer"><code>Iterator</code></a> trait, which has 75(!) methods, each of which come with ownership requirements. How can list comprehensions be translated into uses of a comparable API while still being lifetime-correct?</p>
</li>
<li><p>Given Rust’s exacting control over memory and lifetimes, no data structure can be one-size-fits-all. How can the type of structure produced by a comprehension be controlled by the user? How can the translation of the comprehension accommodate data structures with different requirements, like fixed sizes or custom allocators?</p>
</li>
</ul>
<p>This is not an exhaustive list, but it should give you some appreciation for the challenges involved in Rust’s design.</p>
<h2>Being compiled is not, itself, an obstacle</h2>
<p>In your question, you write</p>
<blockquote>
<p>I know I'm not that strong with compiled languages, but shouldn't we be able to do something like so?</p>
</blockquote>
<p>but the fact that Rust is compiled has little to do with it: lots of languages are compiled. Haskell, which Rust takes a great deal of inspiration from, is compiled to native code in a very traditional way, and it supports comprehensions with great generality. But Haskell, like most programming languages, is not a systems programming language. After all, systems programming is a fairly niche (albeit very important!) domain, so most languages do not choose to expose all of this complexity to the programmer.</p>
<p>But in Rust, asking such things of the programmer is not just expected, it is <em>necessary</em> for the domains Rust was originally designed to target. Given that, avoiding list comprehensions helps to keep the language more orthogonal:</p>
<ul>
<li><p>Constructs in the language that allocate memory are kept few and controlled, so lifetimes are well-defined, and the user has explicit control over where the memory goes and how long it lives.</p>
</li>
<li><p>Other features of the language, such as <code>for</code> loops, can be used to fill in that memory, which can be done in a much more regular way. This avoids coupling allocation and mutation, which permits greater flexibility.</p>
</li>
<li><p>In cases where abstractions that combine these concerns are useful, higher-order functions like <code>map</code> and <code>filter</code> are readily available, and they can be given precisely the types and lifetime signatures they need, on a case-by-case basis.</p>
</li>
</ul>
<h2>Features must pay their weight</h2>
<p>In spite of all these challenges, Rust includes many language features that deal with the complexity. In addition to the most obvious example of pointers and references themselves and the presence of the borrow checker, other examples include closures (with their complement of <code>Fn</code>, <code>FnMut</code>, and <code>FnOnce</code> traits) and <code>async</code>/<code>await</code> (with its careful desugaring into a state machine).</p>
<p>Indeed, several of the pieces one would need to implement comprehensions already exist. Much as Rust has the <code>IntoIterator</code> trait for polymorphically injecting values into an <code>Iterator</code>, it also has the <code>FromIterator</code> trait from extracting results. Together with the <code>Iterator</code> methods <code>collect</code> and <code>collect_into</code>, it is possible to achieve many of the same things one could achieve with comprehensions using the <code>Iterator</code> API.</p>
<p>Although it would be <em>possible</em> to specify comprehensions in terms of a desugaring to these APIs, baking something into the language comes at a high cost, and it is not done lightly. Using <code>collect</code> and <code>collect_into</code> reuses existing language features to accomplish the same end, and some of those (such as explicit type disambiguation using the “turbofish” <code>::&lt;&gt;</code> operator) would not come quite so automatically were comprehensions to be made a language feature.</p>
<p><strong>tl;dr</strong>: The design of certain things in Rust is not easy because it expects you, the programmer, to make more choices, and making these choices available is the purpose of the Rust programming language.</p>

CC BY-SA 4.0
2023-09-01T00:48:36.550Z
2023-09-01T17:22:32.127Z
2023-09-01T17:22:32.127Z
Alexis King
CC BY-SA 4.0
2023-09-01T09:21:04.587Z
C++ manages something similar in `std::ranges::views` namespace, so not entirely impossible for a good systems-programming language.
Toby Speight
CC BY-SA 4.0
2023-09-01T12:34:02.477Z
The `Iterator` and `FromIterator` traits already give programmers control over all of these things. I don't see why list comprehensions would need separate APIs. The only substantial difference I can see between a list comprehension and `.map(...).collect()` is that in `.map(...)` you're explicitly passing a callback function, whereas in a list comprehension it is less obvious whether the term `x + 1` is evaluated in a different stack frame.
kaya3
CC BY-SA 4.0
2023-09-01T12:42:48.580Z
To directly answer each, (1) the memory is allocated by whatever implements `FromIterator`; (2) you control that by calling either `.iter()` or `.into_iter()` on the source; (3) local variables inside the body would necessarily have a lifetime of one iteration, same as for every other loop; values only escape when moved to a different owner, the borrow checker handles this; (4) you don't need new APIs, and [this crate](https://docs.rs/mapcomp/0.2.2/src/mapcomp/lib.rs.html#186) does the translation in about 30 LOC; (5) the same way `.collect()` does (i.e. the `FromIterator` trait).
kaya3
CC BY-SA 4.0
2023-09-01T13:06:17.837Z
This is **wrong**, at multiple levels. First of all, while Rust does allow manual memory management, the abstractions one typically use _automatically_ handle it, so that few developers worry about when and where memory is allocated/deallocated while writing Rust. A bit more than in Haskell -- due to lifetimes -- but not overly much. Certainly not as much as in C. (cont.)
Matthieu M.
CC BY-SA 4.0
2023-09-01T13:07:13.283Z
(cont.) Secondly, as mentioned by kaya3, the `collect()` call allows collecting all elements yielded by an iterator into a collection. `let new_arr: [_; 5] = my_arr.into_iter().map(|x| x + 1).collect();` is a "list comprehension" in Rust. It performs all allocations necessary... it's just lacking the sugar.
Matthieu M.
CC BY-SA 4.0
2023-09-01T15:27:36.307Z
@MatthieuM. The very existence of both `collect` and `collect_into` is more or less precisely my point. I certainly never intended to suggest that the complexity here is identical to C (or even C++!), but these sorts of distinctions are very common in Rust! I am sure Rust *could* support this, if they so chose. But it would require making more choices than simply having `collect` available.
Alexis King
CC BY-SA 4.0
2023-09-01T16:36:48.723Z
@AlexisKing your motivating examples are indeed challenges for implementing comprehensions in Rust, don't they apply equally to basically every other language feature in Rust? Like say, an anonymous closure passed as a callback? It's not exactly clear how comprehensions are fundamentally different...or is your point that everything in Rust is challenging because of the memory management mechanisms and so the bar to adding anything is higher?
Jared Smith
CC BY-SA 4.0
2023-09-01T17:03:02.407Z
@JaredSmith Sure, they’re not fundamentally different. But closures are *essential*—they cannot be implemented as a derived concept—so it makes sense to work them out in spite of the challenges. And indeed, closures are very complex in Rust, between `Fn`, `FnOnce`, `FnMut`, and function pointers. Async/await is another feature with quite a lot of complexity that was deemed worth the effort of implementation. But the question asks why Rust chooses not to support list comprehensions, and the choices one would have to make in doing so are a substantial inhibiting factor.
Alexis King
CC BY-SA 4.0
2023-09-01T17:05:03.857Z
@AlexisKing yeah that makes total sense, I'm just not sure that's what I got from your answer, which is why I asked :)
Jared Smith
CC BY-SA 4.0
2023-09-01T17:05:53.760Z
@JaredSmith Well, I did say they are not fundamentally incompatible. :) But you’re not the only person to have been confused in that way, so perhaps I ought to edit it to be a little bit clearer.
Alexis King
CC BY-SA 4.0
2023-09-01T22:18:11.773Z
@AlexisKing, this is the most nuanced, informative answer I've ever received in the StackExchange universe.  I see what you mean about ownership, as well as the potential memory consumption side effects incurred by operations within the list comprehension loop (thanks for that bit of vocabulary, too).
Philip Grabenhorst
CC BY-SA 4.0
2023-09-01T22:18:18.627Z
Also, thanks @everyone for your patience with my cluelessness These are the kinds of issues I was hoping to discover when getting into Rust!
Philip Grabenhorst
<h3>Python List Comprehensions</h3>
<p>In Python, there are 3 blessed collections -- <code>list</code>, <code>set</code>, and <code>dict</code> -- for which syntactic sugar built into the language allow creating one, or the other, directly. The List Comprehension syntax available in Python allows creating either:</p>
<pre class="lang-python prettyprint-override"><code>new_list = [i + 1 for i in old_list]
new_set = {i + 1 for i in old_set}
new_dict = {k:v+1 for (k, v) in old_dict.items()}
</code></pre>
<p>This is great when you use them... and less so when you'd like another.</p>
<h3>Equality: no second-class citizen in Rust!</h3>
<p>Rust is a Systems Programming Language.</p>
<p>In Python, most developers do not know, nor care, exactly how <code>dict</code> works under the hood: what's the memory layout? The type of hash-map used? The hashing algorithm used?</p>
<p>Systems Programmers, however, do care. They may care due to performance reasons, they may care due to the lack of memory allocation in their domain (embedded, kernel). This means that unlike Python developers, they are <em>much</em> more likely to need specialized collections, rather than the run-of-the-mill standard collections.</p>
<p>And therein lies the rub. The thought of all those specialized collections being second-class citizens, and much more awkward to use than 3 blessed standard collections, did not sit well with Rust developers.</p>
<p>Therefore, instead of creating a list-comprehension syntax that would only be usable with 3 standard collections, they instead focused on making the operations you are looking for work with any collection via Rust's powerful trait system:</p>
<ol>
<li>The <code>Iterator</code> trait allows iterating over anything, and provides many way to filter/transform the elements being yielded.</li>
<li>The <code>FromIterator</code> trait allows any type implementing it to be built from an <code>Iterator</code>.</li>
</ol>
<p>Therefore, it is perfectly possible to create one collection from another:</p>
<pre class="lang-rust prettyprint-override"><code>let new: Vec&lt;_&gt; = [1, 2, 3, 4, 5].into_iter().map(|i| i + 1).collect();
let new: VecDeque&lt;_&gt; = [1, 2, 3, 4, 5].into_iter().map(|i| i + 1).collect();
let new: LinkedList&lt;_&gt; = [1, 2, 3, 4, 5].into_iter().map(|i| i + 1).collect();
</code></pre>
<p><em>Note: It is not possible to <em>directly</em> create an array due to fallibility concerns, though maybe one should submit a RFC to allow collecting into a <code>Result&lt;[_; _], _&gt;</code> as that would handle the fallibility concerns.</em></p>
<p>Iterator chains are very powerful: take, skip, filter, filter_map, enumerate, ... so that list comprehensions would be a step down (they can only map and filter) in many cases.</p>
<p>Is it worth having special syntax for the easy case? Well, so far, the answer from Rust developers has been that they don't feel it would be.</p>
<hr />
<p>On a tangent, the same iterators chains are available for <em>extending</em> an existing collection with more elements:</p>
<pre class="lang-rust prettyprint-override"><code>my_collection.extend([1, 2, 3, 4, 5].into_iter().map(|i| i + 1));
</code></pre>
<p>The <code>extend</code> method takes any iterator yielding elements of the same type as the collection.</p>
<h3>Macros!</h3>
<p>It should also be noted that Rust has macros.</p>
<p>A quick search on <a href="https://crates.io" rel="noreferrer">crates.io</a> reveals that some Rust users felt the need for quick &amp; easy, and they went ahead and created macros for it, such as the <a href="https://crates.io/crates/list_comprehension_macro" rel="noreferrer"><code>list_comprehension_macro</code></a> crate (and a bunch of others). From the README:</p>
<pre class="lang-rust prettyprint-override"><code>let even_squares = comp![x.pow(2) for x in arr if x % 2 == 0];
let flatten_matrix = comp![x for row in arr for x in row];
let dict_comp = comp!{x: x.len() for x in arr};
</code></pre>
<p>This, in turn, is also a disincentive to include any such functionality in the standard library, as it's easy enough to make your own, or use a 3rd-party library.</p>
<p><em>Note: a somewhat minimal standard library is also an explicit goal of Rust developers.</em></p>

CC BY-SA 4.0
2023-09-01T13:33:20.800Z
2023-09-01T15:06:01.413Z
2023-09-01T15:06:01.413Z
Matthieu M.
CC BY-SA 4.0
2023-09-01T14:03:20.867Z
So the Macros demonstrate that for/list comprehensions in Rust are possible - in contradiction to the original question and answer https://langdev.stackexchange.com/a/2851/2163
Christian Lindig
CC BY-SA 4.0
2023-09-01T14:05:13.343Z
There are more than two in Python: Python also has set comprehensions, and arguably generator expressions should count ─ they create an iterator object, although not a collection.
kaya3
CC BY-SA 4.0
2023-09-01T14:11:05.687Z
There is also no need for a fallible version of the `FromIterator` trait, since `Result` implements `FromIterator` in a suitable way; see [here](https://doc.rust-lang.org/std/result/enum.Result.html#method.from_iter) and [this prior feature request](https://github.com/rust-lang/rfcs/issues/1839).
kaya3
CC BY-SA 4.0
2023-09-01T15:02:28.227Z
@ChristianLindig: It's possible indeed, just not built-in.
Matthieu M.
CC BY-SA 4.0
2023-09-01T15:05:14.890Z
@kaya3: I forgot set expressions, I will amend the answer. Do note that I did not speak about the need for a fallible version of `FromIterator`, just that there's today no fallible implementation of `collect` for a collection in the standard library, as they all grow as needed, and a fixed-size array would be the first such implementation. I think it may require specialization to make it work, notably, since `Result` is the outer type.
Matthieu M.
CC BY-SA 4.0
2023-09-01T15:22:56.770Z
For what it’s worth, fallible `collect` *is* available in nightly as [`try_collect`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.try_collect).
Alexis King
CC BY-SA 4.0
2023-09-01T16:32:00.023Z
@AlexisKing: I don't think that's the same thing. `try_collect` is about propagating the "failure" from the elements out, AFAIK, and won't work for collecting `Iterator<Item = i32>` into an `[i32; 5]`.
Matthieu M.
CC BY-SA 4.0
2023-09-01T16:57:45.240Z
@MatthieuM. Ah, you’re right, I skimmed your answer and misinterpreted the sort of fallibility you were referring to. (I upvoted it.)
Alexis King
CC BY-SA 4.0
2023-09-02T13:26:01.970Z
I'm not following the argument about blessed types. The generic "comprehension" form in Python is the generator expression that one can throw into *arbitrary* constructors, e.g. `deque(i + 1 for i in old_list)`. `[` `]`/`{:` `}`/`{` `}` are sugar for list/dict/set constructors by themselves. Semantically, a list/dict/set comprehension is just combining an *already blessed* constructor with a generator expression. Comprehensions aren't what blesses list/dict/set, nor is blessing list/dict/set needed for comprehensions.
MisterMiyagi
CC BY-SA 4.0
2023-09-02T13:32:22.527Z
@MisterMiyagi: My understanding was that the OP was talking about the likes of list comprehensions, and not about generators, and thus I explained why there's no list comprehension (or similar) in Rust. You are indeed correct that generator expressions would not conflict with the idea of no second class citizen.
Matthieu M.
CC BY-SA 4.0
2023-09-06T16:06:05.583Z
@ChristianLindig the big difference between a core language feature and a macro in a library is that people expect core language features to work everywhere. A macro in a library (potentially even the standard library, since rust's standard library is now split into multiple sections with different assumptions) has fewer such concerns.
Peter Green
# Why does Rust choose not to provide `for` comprehensions?
Posted by: username_0 on 2023-08-31
Okay, so this might be a silly question. I'm a big fan of functional programming paradigms in higher-level scripting languages, so I'm probably using Rust wrong. However, I'm new to Rust and I'm curious why this isn't a thing. If someone could explain why this isn't a language feature (or provide a solid technical explanation for why it's a bad idea), I'd appreciate it!
So, I'm working my way through the Rust Book. So far, I'm on board with the "evaluating a block as an expression given its last line is an expression business." It seems kind of LISP-y or Swift-y to me. Anyway, Rust lets you do things like this...
let x = {
    let tmp: u32 = 1;
    tmp + 1
};

println!("The value of x is {x}");
// the value is 2

You can also loop over an array, providing a block where you can perform an operation with each element...
let my_arr = [1, 2, 3, 4, 5];

for item in my_arr {
    println!("The value of item is {item}");
}
// the value of item is 1-5

So... given the block expression rules above (coupled with Rust's seeming incredible power to infer strictly typed values), why is it that we can't use for x in y as a way to produce a mapped array of values? I know I'm not that strong with compiled languages, but shouldn't we be able to do something like so?
let my_arr = [1, 2, 3, 4, 5];

let new_arr = for item in my_arr { item + 1 };

for item in new_arr {
    println!("The value of item is {item}");
}
// ???

... which, in my little Rust head-canon here, should result in a new array of values [2, 3, 4, 5, 6]. It's my assumption that, if we can infer the value of variable x in the first example, we should be able to infer the member values of the array new_arr and provision for it, accordingly. Is there a reason why this wouldn't work in Rust? Is it contrary to some design principle of the language that I've yet to discover?
**Tags:** rust foreach-loop
> **Comments:**
> **username_1**: The reason it doesn't work is because, to use something in a for loop it specifically needs to implement `IntoIter` which consumes self. Some methods implement an `iter()` method that only borrows `self` and allows you to iterate over it multiple times but there is no trait for that and for loops will never use it unless you call it explicitly.  
> **username_2**: @mousetail Remember not to answer with comments!  
> **username_1**: @RydwolfPrograms It doesn't really answer the question, the question is if it could be done in a different way, not why it currently works this way  
> **username_3**: I don't understand why the two other things (block expressions and `for` loops) would suggest that the third thing (list comprehensions) should work. The value of a `for` loop is `()`, i.e. the unit value, so `let new_arr = for ...` will declare a variabel `new_arr` with the value `()`.  
> **username_4**: Note that what you're calling a "closure" in the question is actually called a "block". Closures in rust have a different meaning.  
> **username_3**: A block expression isn't a closure in any sense, this seems to be a misconception.  
> **username_1**: You can use `.iter().map(|x|x+1)` to perform the effect you want  
---
## 4 answers
### Answer by username_7 on 2023-09-01. Score: 34 | Accepted answer
# Manual memory management is complicated

Rust is a [systems programming](https://en.wikipedia.org/wiki/Systems_programming) language. While it is very much *possible* to use Rust for general-purpose application development, it is primarily specialized to the task of low-level systems programming. The most obvious example of this is its use of manual memory management.

Since Rust aims to achieve full [memory safety](https://en.wikipedia.org/wiki/Memory_safety) within its safe subset, much of the language is built around structured mechanisms for controlling memory allocation and lifetimes. In many languages, these concerns are largely handled by the programming language, which makes it both possible and easy to, say, materialize a value from nothing and not worry about where it’s being allocated, who owns it, and how long it lives. The answers to those questions are both simple and uniform: the value is allocated in the heap, the runtime owns its memory, and it lives until it is no longer reachable. Rust does not have such luxuries, and this does not just mean that things are less automatic, it also means they are more *nuanced*.

In Rust, one can allocate memory in numerous different ways, and values’ lifetimes can be controlled by any of several different mechanisms. There are many different gradations of ownership, each with their own implications and tradeoffs. This is not a shortcoming of the language, it’s the *point*: people doing systems programming care about the distinctions between the potential options. The purpose of Rust is to give such programmers low-level control over these mechanisms while retaining safety, and in exchange, the programmer must be exquisitely explicit about each of the choices they make.

[List comprehensions](https://en.wikipedia.org/wiki/List_comprehension) are not *fundamentally* incompatible with this attitude, but they are also far from straightforward. Here are a few examples of questions they raise:

* Where does the memory for the list returned by a comprehension come from? How long does it live for? What happens when it is freed?
* If the programmer is mapping over an existing structure, is the structure’s memory reused, consuming it in the process? If so, how can the language ensure that memory is not inadvertently accessed from within the body of the loop while the structure is somehow in an invalid state?
* Since values can escape the body of a list comprehension, how is the memory for local variables within the loop body managed? Can the space used by the loop be linear in the number of iterations? Do values need to be copied at the end of each iteration?
* In Rust, iteration is overloadable via the [`Iterator`](https://doc.rust-lang.org/std/iter/trait.Iterator.html) trait, which has 75(!) methods, each of which come with ownership requirements. How can list comprehensions be translated into uses of a comparable API while still being lifetime-correct?
* Given Rust’s exacting control over memory and lifetimes, no data structure can be one-size-fits-all. How can the type of structure produced by a comprehension be controlled by the user? How can the translation of the comprehension accommodate data structures with different requirements, like fixed sizes or custom allocators?

This is not an exhaustive list, but it should give you some appreciation for the challenges involved in Rust’s design.

## Being compiled is not, itself, an obstacle

In your question, you write

> I know I'm not that strong with compiled languages, but shouldn't we be able to do something like so?

but the fact that Rust is compiled has little to do with it: lots of languages are compiled. Haskell, which Rust takes a great deal of inspiration from, is compiled to native code in a very traditional way, and it supports comprehensions with great generality. But Haskell, like most programming languages, is not a systems programming language. After all, systems programming is a fairly niche (albeit very important!) domain, so most languages do not choose to expose all of this complexity to the programmer.

But in Rust, asking such things of the programmer is not just expected, it is *necessary* for the domains Rust was originally designed to target. Given that, avoiding list comprehensions helps to keep the language more orthogonal:

* Constructs in the language that allocate memory are kept few and controlled, so lifetimes are well-defined, and the user has explicit control over where the memory goes and how long it lives.
* Other features of the language, such as `for` loops, can be used to fill in that memory, which can be done in a much more regular way. This avoids coupling allocation and mutation, which permits greater flexibility.
* In cases where abstractions that combine these concerns are useful, higher-order functions like `map` and `filter` are readily available, and they can be given precisely the types and lifetime signatures they need, on a case-by-case basis.

## Features must pay their weight

In spite of all these challenges, Rust includes many language features that deal with the complexity. In addition to the most obvious example of pointers and references themselves and the presence of the borrow checker, other examples include closures (with their complement of `Fn`, `FnMut`, and `FnOnce` traits) and `async`/`await` (with its careful desugaring into a state machine).

Indeed, several of the pieces one would need to implement comprehensions already exist. Much as Rust has the `IntoIterator` trait for polymorphically injecting values into an `Iterator`, it also has the `FromIterator` trait from extracting results. Together with the `Iterator` methods `collect` and `collect_into`, it is possible to achieve many of the same things one could achieve with comprehensions using the `Iterator` API.

Although it would be *possible* to specify comprehensions in terms of a desugaring to these APIs, baking something into the language comes at a high cost, and it is not done lightly. Using `collect` and `collect_into` reuses existing language features to accomplish the same end, and some of those (such as explicit type disambiguation using the “turbofish” `::<>` operator) would not come quite so automatically were comprehensions to be made a language feature.

**tl;dr**: The design of certain things in Rust is not easy because it expects you, the programmer, to make more choices, and making these choices available is the purpose of the Rust programming language.
> **Comments:**
> **username_8** (2023-09-01): C++ manages something similar in `std::ranges::views` namespace, so not entirely impossible for a good systems-programming language.  
> **username_3** (2023-09-01): The `Iterator` and `FromIterator` traits already give programmers control over all of these things. I don't see why list comprehensions would need separate APIs. The only substantial difference I can see between a list comprehension and `.map(...).collect()` is that in `.map(...)` you're explicitly passing a callback function, whereas in a list comprehension it is less obvious whether the term `x + 1` is evaluated in a different stack frame.  
> **username_3** (2023-09-01): To directly answer each, (1) the memory is allocated by whatever implements `FromIterator`; (2) you control that by calling either `.iter()` or `.into_iter()` on the source; (3) local variables inside the body would necessarily have a lifetime of one iteration, same as for every other loop; values only escape when moved to a different owner, the borrow checker handles this; (4) you don't need new APIs, and [this crate](https://docs.rs/mapcomp/0.2.2/src/mapcomp/lib.rs.html#186) does the translation in about 30 LOC; (5) the same way `.collect()` does (i.e. the `FromIterator` trait).  
> **username_9** (2023-09-01): This is **wrong**, at multiple levels. First of all, while Rust does allow manual memory management, the abstractions one typically use _automatically_ handle it, so that few developers worry about when and where memory is allocated/deallocated while writing Rust. A bit more than in Haskell -- due to lifetimes -- but not overly much. Certainly not as much as in C. (cont.)  
> **username_9** (2023-09-01): (cont.) Secondly, as mentioned by kaya3, the `collect()` call allows collecting all elements yielded by an iterator into a collection. `let new_arr: [_; 5] = my_arr.into_iter().map(|x| x + 1).collect();` is a "list comprehension" in Rust. It performs all allocations necessary... it's just lacking the sugar.  
> **username_7** (2023-09-01): @MatthieuM. The very existence of both `collect` and `collect_into` is more or less precisely my point. I certainly never intended to suggest that the complexity here is identical to C (or even C++!), but these sorts of distinctions are very common in Rust! I am sure Rust *could* support this, if they so chose. But it would require making more choices than simply having `collect` available.  
> **username_10** (2023-09-01): @AlexisKing your motivating examples are indeed challenges for implementing comprehensions in Rust, don't they apply equally to basically every other language feature in Rust? Like say, an anonymous closure passed as a callback? It's not exactly clear how comprehensions are fundamentally different...or is your point that everything in Rust is challenging because of the memory management mechanisms and so the bar to adding anything is higher?  
> **username_7** (2023-09-01): @JaredSmith Sure, they’re not fundamentally different. But closures are *essential*—they cannot be implemented as a derived concept—so it makes sense to work them out in spite of the challenges. And indeed, closures are very complex in Rust, between `Fn`, `FnOnce`, `FnMut`, and function pointers. Async/await is another feature with quite a lot of complexity that was deemed worth the effort of implementation. But the question asks why Rust chooses not to support list comprehensions, and the choices one would have to make in doing so are a substantial inhibiting factor.  
> **username_10** (2023-09-01): @AlexisKing yeah that makes total sense, I'm just not sure that's what I got from your answer, which is why I asked :)  
> **username_7** (2023-09-01): @JaredSmith Well, I did say they are not fundamentally incompatible. :) But you’re not the only person to have been confused in that way, so perhaps I ought to edit it to be a little bit clearer.  
> **username_0** (2023-09-01): @AlexisKing, this is the most nuanced, informative answer I've ever received in the StackExchange universe.  I see what you mean about ownership, as well as the potential memory consumption side effects incurred by operations within the list comprehension loop (thanks for that bit of vocabulary, too).  
> **username_0** (2023-09-01): Also, thanks @everyone for your patience with my cluelessness These are the kinds of issues I was hoping to discover when getting into Rust!  
---
### Answer by username_9 on 2023-09-01. Score: 15
Python List Comprehensions
In Python, there are 3 blessed collections -- list, set, and dict -- for which syntactic sugar built into the language allow creating one, or the other, directly. The List Comprehension syntax available in Python allows creating either:
new_list = [i + 1 for i in old_list]
new_set = {i + 1 for i in old_set}
new_dict = {k:v+1 for (k, v) in old_dict.items()}

This is great when you use them... and less so when you'd like another.
Equality: no second-class citizen in Rust!
Rust is a Systems Programming Language.
In Python, most developers do not know, nor care, exactly how dict works under the hood: what's the memory layout? The type of hash-map used? The hashing algorithm used?
Systems Programmers, however, do care. They may care due to performance reasons, they may care due to the lack of memory allocation in their domain (embedded, kernel). This means that unlike Python developers, they are much more likely to need specialized collections, rather than the run-of-the-mill standard collections.
And therein lies the rub. The thought of all those specialized collections being second-class citizens, and much more awkward to use than 3 blessed standard collections, did not sit well with Rust developers.
Therefore, instead of creating a list-comprehension syntax that would only be usable with 3 standard collections, they instead focused on making the operations you are looking for work with any collection via Rust's powerful trait system:

The Iterator trait allows iterating over anything, and provides many way to filter/transform the elements being yielded.
The FromIterator trait allows any type implementing it to be built from an Iterator.

Therefore, it is perfectly possible to create one collection from another:
let new: Vec<_> = [1, 2, 3, 4, 5].into_iter().map(|i| i + 1).collect();
let new: VecDeque<_> = [1, 2, 3, 4, 5].into_iter().map(|i| i + 1).collect();
let new: LinkedList<_> = [1, 2, 3, 4, 5].into_iter().map(|i| i + 1).collect();

Note: It is not possible to directly create an array due to fallibility concerns, though maybe one should submit a RFC to allow collecting into a Result<[_; _], _> as that would handle the fallibility concerns.
Iterator chains are very powerful: take, skip, filter, filter_map, enumerate, ... so that list comprehensions would be a step down (they can only map and filter) in many cases.
Is it worth having special syntax for the easy case? Well, so far, the answer from Rust developers has been that they don't feel it would be.

On a tangent, the same iterators chains are available for extending an existing collection with more elements:
my_collection.extend([1, 2, 3, 4, 5].into_iter().map(|i| i + 1));

The extend method takes any iterator yielding elements of the same type as the collection.
Macros!
It should also be noted that Rust has macros.
A quick search on crates.io reveals that some Rust users felt the need for quick & easy, and they went ahead and created macros for it, such as the list_comprehension_macro crate (and a bunch of others). From the README:
let even_squares = comp![x.pow(2) for x in arr if x % 2 == 0];
let flatten_matrix = comp![x for row in arr for x in row];
let dict_comp = comp!{x: x.len() for x in arr};

This, in turn, is also a disincentive to include any such functionality in the standard library, as it's easy enough to make your own, or use a 3rd-party library.
Note: a somewhat minimal standard library is also an explicit goal of Rust developers.
> **Comments:**
> **username_11** (2023-09-01): So the Macros demonstrate that for/list comprehensions in Rust are possible - in contradiction to the original question and answer https://langdev.stackexchange.com/a/2851/2163  
> **username_3** (2023-09-01): There are more than two in Python: Python also has set comprehensions, and arguably generator expressions should count ─ they create an iterator object, although not a collection.  
> **username_3** (2023-09-01): There is also no need for a fallible version of the `FromIterator` trait, since `Result` implements `FromIterator` in a suitable way; see [here](https://doc.rust-lang.org/std/result/enum.Result.html#method.from_iter) and [this prior feature request](https://github.com/rust-lang/rfcs/issues/1839).  
> **username_9** (2023-09-01): @ChristianLindig: It's possible indeed, just not built-in.  
> **username_9** (2023-09-01): @kaya3: I forgot set expressions, I will amend the answer. Do note that I did not speak about the need for a fallible version of `FromIterator`, just that there's today no fallible implementation of `collect` for a collection in the standard library, as they all grow as needed, and a fixed-size array would be the first such implementation. I think it may require specialization to make it work, notably, since `Result` is the outer type.  
> **username_7** (2023-09-01): For what it’s worth, fallible `collect` *is* available in nightly as [`try_collect`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.try_collect).  
> **username_9** (2023-09-01): @AlexisKing: I don't think that's the same thing. `try_collect` is about propagating the "failure" from the elements out, AFAIK, and won't work for collecting `Iterator<Item = i32>` into an `[i32; 5]`.  
> **username_7** (2023-09-01): @MatthieuM. Ah, you’re right, I skimmed your answer and misinterpreted the sort of fallibility you were referring to. (I upvoted it.)  
> **username_12** (2023-09-02): I'm not following the argument about blessed types. The generic "comprehension" form in Python is the generator expression that one can throw into *arbitrary* constructors, e.g. `deque(i + 1 for i in old_list)`. `[` `]`/`{:` `}`/`{` `}` are sugar for list/dict/set constructors by themselves. Semantically, a list/dict/set comprehension is just combining an *already blessed* constructor with a generator expression. Comprehensions aren't what blesses list/dict/set, nor is blessing list/dict/set needed for comprehensions.  
> **username_9** (2023-09-02): @MisterMiyagi: My understanding was that the OP was talking about the likes of list comprehensions, and not about generators, and thus I explained why there's no list comprehension (or similar) in Rust. You are indeed correct that generator expressions would not conflict with the idea of no second class citizen.  
> **username_13** (2023-09-06): @ChristianLindig the big difference between a core language feature and a macro in a library is that people expect core language features to work everywhere. A macro in a library (potentially even the standard library, since rust's standard library is now split into multiple sections with different assumptions) has fewer such concerns.  
---
### Answer by username_3 on 2023-08-31. Score: 6
Note: this answer mainly addresses the question of why for loops are not list comprehensions in Rust. The question title was edited later to be about having list comprehensions at all.

Why don't for loops evaluate to arrays?
The feature you're looking for is called list comprehensions. For example, Python has them:
new_arr = [x + 1 for x in my_arr]

Note that even in languages which have this feature, it is not the same as a for loop:
# does nothing
for x in my_arr: x + 1

# syntax error
new_arr = for x in my_arr: x + 1

This makes sense because a for loop is a statement written for its side-effects, whereas a list comprehension is an expression used to generate a list, and the individual terms in the list comprehension are generally recommended to have no side-effects. So these are two different language constructs for two different purposes and there is no need to unify them.
The fact that Rust is an expression-based language (i.e. for loops and blocks are expressions which produce values) doesn't change this; the vast majority of the time you use a for loop, it's not to create a new mapped array, so it would be strange (and possibly wasteful) if this was the behaviour of all for loops.
It would also be problematic for a statically typed language to make every for loop a list comprehension. For example, if the for loop's type was an array or some other sequence, then the two branches of the conditional expression like this one would not share a type:
if xs.is_empty() {
    // type is ()
    println!("No elements");
} else {
    // type is an array of ()
    for x in xs.iter() {
        println!("{x}");
    }
}

So it's better in the much more common case, for the type of for expressions to be (). If you want a mapped array then you can still achieve that by different means.

What about something other than for loops?
So then, why doesn't Rust have a different language construct for list comprehensions? Well, you can already use iterators, .map() and .collect() to build a mapped data structure of any type implementing the FromIterator trait, so the only thing that list comprehensions would add would be syntax sugar for those standard library features.
Such syntax sugar might be valuable, but Rust may choose not to have it because of the language's philosophy that the standard library should not be privileged over third-party libraries any more than necessary; and because syntax sugar can be added by third-party code using macros (e.g. this crate) anyway if some users want it.
> **Comments:**
> **username_5** (2023-09-01): **Comments have been [moved to chat](https://chat.stackexchange.com/rooms/148242/discussion-on-answer-by-kaya3-why-does-rust-choose-not-to-provide-for-comprehe); please do not continue the discussion here.** Before posting a comment below this one, please review the [purposes of comments](/help/privileges/comment). Comments that do not request clarification or suggest improvements usually belong as an [answer](/help/how-to-answer), on [meta], or in [chat]. Comments continuing discussion may be removed.  
---
### Answer by username_6 on 2023-09-01. Score: 5
Maybe because I am not a functional language guy, I don't see at all your proposed syntax as obvious. But if you were to submit an RFC proposal with such thing, the first question that comes to my mind is: what should be the type returned by `for`?

Neither of the obvious choice seems good:

1. it can't return an array, because arrays have fixed size, and `for` iterates over a type implementing the trait `Iterator`, but a generic `Iterator` doesn't know its size at compile time. Nor at runtime. Heck, it doesn't even have to terminate, it is perfectly valid for an iterator to be infinite (turning a `for` into an infinite loop). Thus, array is out of question.
2. it could return a `Vec`: if the compiler detects the `for` return value is being used, it creates a `Vec` and inserts the output of every iteration. But `Vec` is a high level library class, it doesn't bode well for a fundamental language construct like `for` to create and return a `Vec`. But what is worse is that `Vec` is not even available for every rust program, because it allocates memory. If you mark your program as `#[no_std]` (which is used for bare metal programming and other limited resources environments), it won't have a default global allocator, and `Vec` and anything that allocates memory won't be available.
3. it could return an inferred `FromIterator` type, like `Iterator::collect()` does. Then you could use any container you like, including some that doesn't need to allocate for `#[no_std]` environments. To avoid allocation, each `for` iteration would have to be executed on demand from the `FromIterator::from_iter()` function. I think this completely subverts the distinction between `for` and an iterator (`for` has to be effectively turned into an iterator for this to work), and violates the principle of least surprise. We expect a `for` to be greedy and execute right now, while we expect an iterator to be lazy and only produce the next element when called, if ever. In this case, each `for` iteration would have to be called from the `FromIterator` implementation, and executed in between whatever code is in there, including the possibility of side effects, because this can be an user supplied function. It can even avoid running the iterator entirely, and then your `for` simply does not execute.

Given the return type difficulty, the non-obviousness of the construct (for me at least, but I believe I am representative of the average rust programmer), and most importantly, the readily available alternatives of `my_arr.map(...)` or even `any_iterable.iter().map(...).collect()` (which is basically solution 3), I doubt this was ever seriously considered.
---
