<p>Many languages include a &quot;raw string&quot; literal type, intended for arbitrary (or as close as possible to arbitrary) strings of characters (and/or bytes), ignoring escape sequences and the like. This can be handy for things like regular expressions, where using typical string literals would require unreadable and bug-inviting double escaping (e.g., <code>&quot;a\\?\\\\b&quot;</code> for a regex matching <code>a?\b</code>).</p>
<p>I've seen a ton of variations in raw string syntax, with varying capabilities and readability:</p>
<ul>
<li>JavaScript: <code>String.raw`&quot;I'm tired of escaping things like '\'&quot;`</code></li>
<li>Python: <code>r&quot;I can include \ without worries!&quot;</code></li>
<li>Lua: <code>[[I can hold arbitrary strings! &quot;'\[] ]]</code></li>
<li>Rust: <code>r#&quot;String includes quotes? No worries, wrap it in potato shreds&quot;#</code></li>
</ul>
<p>What are the advantages of these options over one another, or other alternatives?</p>

<syntax><strings>
What are some different approaches to raw string syntax, and what are their pros and cons?
CC BY-SA 4.0
2023-09-12T12:34:50.293Z
Does Rust also allow `##"multiple hashes"##` like in Swift?
Bbrk24
CC BY-SA 4.0
2023-09-12T13:03:32.137Z
@Bbrk24 yes, any number
rydwolf
CC BY-SA 4.0
2023-09-12T17:12:27.303Z
Does putting your raw string in a separate file and calling `file_get_contents()` count? Not sure whether that's worth adding as an answer.
TRiG
CC BY-SA 4.0
2023-09-12T19:17:55.687Z
@TRiG Unless it was a feature built into the language speciifcally for this purpose, I don't think it'd really be an on-topic answer
rydwolf
CC BY-SA 4.0
2023-09-20T05:42:26.663Z
Some relevant discussion here: https://github.com/rust-lang/rust/issues/9411
apropos
CC BY-SA 4.0
2024-08-21T15:57:03.790Z
And one type that everyone's missed because it's prehistoric (and not very useful): Hollerith constants. Dating from early versions of Fortran (early 1960s), here's an example of a raw string: `22HThe next 22 characters`.
k314159
CC BY-SA 4.0
2024-08-24T13:26:22.277Z
@k314159 You could make that an answer
rydwolf
<p>I see no mention of Zig so far, so I think it's worth mentioning it as another point in the space.</p>
<p>Zig has regular inline strings with escape sequences like most languages, and <a href="https://ziglang.org/documentation/master/#Multiline-String-Literals" rel="noreferrer">multiline strings</a> which have no escapes and require each line to be prefixed with <code>\\</code>.</p>
<pre><code>const hello_world_in_c =
    \\#include &lt;stdio.h&gt;
    \\
    \\int main(int argc, char **argv) {
    \\    printf(&quot;hello world\n&quot;);
    \\    return 0;
    \\}
;
</code></pre>
<p>Although this means that you cannot paste in a block of text directly as a &quot;raw string&quot; without prefixing the lines, it has some advantages:</p>
<ul>
<li>The grammar remains context-free, unlike Swift, Rust, and others which require matching but arbitrary open/close pairs.</li>
<li>It's impossible to forget to close the string, so in theory this means less catastrophic parse/highlighting failures in an IDE with incremental parsing of broken code.</li>
<li>As <a href="https://matklad.github.io/2023/08/01/on-modularity-of-lexical-analysis.html#Minimized-Example-Lexical-Analysis" rel="noreferrer">Alex Kladov notes</a>, it's possible to lex the file looking at chunks in parallel or incrementally, since the tokens on one line can never affect the tokens on another line.</li>
</ul>

CC BY-SA 4.0
2023-09-16T11:36:44.627Z
2023-09-16T11:36:44.627Z
Ken Micklas
<h1>Swift: Raw strings with interpolation</h1>
<p>Swift, like Rust, uses hash signs for raw strings. However, you can also interpolate into raw strings by adding the hashes to the interpolation marker:</p>
<pre class="lang-swift prettyprint-override"><code>&quot;Regular string interpolation \(x)&quot;
#&quot;Raw string with &quot;quotes&quot; in it&quot;#
#&quot;Raw string with interpolation \#(x)&quot;#
##&quot;Not an interpolation \#(x)&quot;##
##&quot;But this is \##(x)&quot;##
</code></pre>
<p>You can have arbitrarily many hash signs in the literal, but the opening and closing quote marks must match.</p>

CC BY-SA 4.0
2023-09-12T12:40:34.130Z
2023-09-12T12:40:34.130Z
Bbrk24
CC BY-SA 4.0
2024-08-10T17:39:01.783Z
C# also went this route for its design. I love the way Mads Torgersen put phrased it [in this talk](https://youtu.be/v8bqAm4aUFM&t=3170): "You can always win. It might not look nice by the time you win, ... [but you can always win]" You can take any arbitrary string (including other Swift/Rust/C# source code that uses this feature), paste it in directly, and change only the delimiters to make it correct. You don't have to edit the content to appease the host language, and that's awesome.
Alexander
<h1>Go style: backticks only</h1>
<p>Similar to JavaScript, Go uses a pair of backticks for raw strings:</p>
<pre class="lang-go prettyprint-override"><code>&quot;regular&quot;
`raw`

&quot;regular\n    indented\newlines&quot;
`raw
    indented
newlines`
</code></pre>
<p>Which is exceptionally useful for regexes, as you don't need to escape anything, except for the usual escaping of special regex characters:</p>
<pre class="lang-go prettyprint-override"><code>regexp.MustCompile(`(\w)(\w+)(\w)`)
</code></pre>
<h2>Pros</h2>
<ul>
<li>Extremely simple, no new parsing required (just an additional token kind)</li>
<li>Can be used anywhere a regular string literal is used</li>
<li>You cannot escape anything in these, so <code>\n</code> is the literal characters <code>\</code> and <code>n</code>.</li>
</ul>
<h2>Cons</h2>
<ul>
<li><p><strong>You cannot escape anything in these.</strong> This leads to backticks being the only disallowed character because it would end the literal, and you can't escape it because you can't escape a raw string. This can be worked around by concatenating with a regular string literal:</p>
<pre class="lang-go prettyprint-override"><code>query := `SELECT * FROM` + &quot;`table`&quot;
</code></pre>
</li>
<li><p>You need to keep track of indention level relative to the start of the raw string for indented parts to have the correct indentation.</p>
</li>
</ul>

CC BY-SA 4.0
2023-06-28T13:29:49.987Z
2024-08-05T00:12:31.890Z
2024-08-05T00:12:31.890Z
bigyihsuan
CC BY-SA 4.0
2024-08-06T10:11:19.743Z
The cons can be resolved as follows: allow more than one backtick, and require the end delimiter to match the begin delimiter.
Thomas Mueller
CC BY-SA 4.0
2024-08-08T19:14:55.337Z
@ThomasMueller This still has the problem that the string cannot start with a backtick.
xigoi
CC BY-SA 4.0
2024-08-09T20:39:47.750Z
Right. That would require using a multi-line raw string: the first line contains two backticks (as the start delimiter), then the second line would contain one backtick, and the last line two backticks (as the end delimiter).
Thomas Mueller
<h1>Python style</h1>
<p>Pros: Low friction syntax-wise, close enough to existing string syntax<br />
Cons: being closed by a regular quotechar still requires you to include a \ for including a quote value</p>
<h1>JavaScript style</h1>
<p>Probably the most confusing of all the syntaxes, a very special case in parsing, requires you to recognize <code>String.raw`str`</code> as a function call.</p>
<h1>Lua style</h1>
<p>Possible confusion with array syntax in other languages  (although in lua's case Arrays are defined by {} so not in this particular case)</p>
<h1>Rust style</h1>
<p>Maybe the most robust of all syntaxes, it's far enough from other rust syntaxes to be confusing or to be accidentally closed by another value, can't think of a downside</p>
<h1>C++</h1>
<p>I'd like to add another case of raw strings: C++. It allows the user to define their own syntax for their specific use case, for example</p>
<pre><code>auto rawstr = R&quot;rawstring(
text
text
text
)rawstring&quot;;

auto rawstr2 = R&quot;aaaaa(
text
text
text
aaaaa)&quot;;
</code></pre>
<p>are both raw strings, and the user can choose a termination value that does not conflict with their expected values and for which the start/end of the string is clearly readable.</p>

CC BY-SA 4.0
2023-06-28T04:52:55.583Z
2023-06-28T05:06:45.060Z
2023-06-28T05:06:45.060Z
kouta-kun
CC BY-SA 4.0
2023-06-28T05:45:48.653Z
I used the C++-like approach in my programming language: https://flatassembler.github.io/AEC_specification.html#multiline_strings
FlatAssembler
CC BY-SA 4.0
2023-09-12T14:34:08.750Z
For those not familiar with the given languages, it would be nice to have an example, each time.
Matthieu M.
CC BY-SA 4.0
2023-09-12T16:33:32.247Z
@MatthieuM. I only added examples for C++ because it's the only one that the original answer didn't have (as they have examples for the ones they do know about)
kouta-kun
<h1>Repeat Quote</h1>
<p>Repeat quote three times or more as <code>STRING_QUOTE_TOKEN</code> until the same token is encountered.</p>
<p>Should report a parse error if not found.</p>
<p>Similar to <code>`</code> in markdown</p>
<pre><code>'''raw string'''

&quot;&quot;&quot;&quot;
multiline * 4
raw string
&quot;&quot;&quot;&quot;
</code></pre>
<p>The disadvantage is that it is a bit long, at least 3 characters.</p>
<p>The advantage is that you don't need to consider any escaping, you can even dynamically interprete escape.</p>
<p>For simplicity, you can also specify that <code>'raw'</code> is the raw string <code>'''raw'literal'''</code> is used to deal with the occurrence of <code>'</code> in the raw string.</p>
<p>The fatal problem is what to do if your first character is <code>'</code>, at this point you have to choose another character like <code>&quot;</code> to form.</p>
<hr />
<p>The dilemma with escaping is that the more complex your escape, the more complex the unescaped form used to express that character.</p>
<p>If you have written <code>tmLanguage</code>, you may have encountered the situation where you have to write <code>\\\\\\\\</code>.</p>
<p>Therefore, the best way is not to escape, and directly capture the content between the two tokens as a raw string.</p>

CC BY-SA 4.0
2023-06-28T05:58:53.493Z
2023-06-28T06:05:13.430Z
2023-06-28T06:05:13.430Z
Aster
CC BY-SA 4.0
2023-09-17T10:16:42.053Z
"you don't need to consider escaping" tell that a code generator for the language. I remember having to write ""${""}" in Scala code generation for Scala code to generate raw strings. But it's less of an issue.
feldentm
CC BY-SA 4.0
2024-08-05T10:28:10.913Z
My language Papyri does this. The "fatal problem" is not just when you want to start a string with a quote, but also when you want to end with one. I am considering delimiters like `~'''` and `'''~` where the `~` symbol is optional on each side independently, and strips whitespace from that side; this would also mean you can use whitespace for alignment purposes at the start and end of multi-line strings.
kaya3
<h2>Recognizing the delimiter</h2>
<p>The examples given in the question obscure an important mechanical detail relevant to Python's implementation (and possibly others). A Python raw string literal <em>uses the same lexing/tokenization rule</em> as other string literals: a string consists of sequences of (anything except the closing quote) <em>or</em> (a backslash followed by a single character). It only has different rules for <em>interpreting</em> that token (everything is treated literally, including backslashes).</p>
<p>This has two important consequences:</p>
<ol>
<li><p>A Python raw string literal <em>can</em> including the closing quote character, <em>but only immediately after</em> a backslash (which the resulting string will contain literally).</p>
</li>
<li><p>A Python raw string literal <em>cannot</em> end with <em>an odd number of</em> backslash characters.</p>
</li>
</ol>
<p>An alternative rule (apparently used in Go, as seen in bigyihsuan's answer) is to just lex the string a character at a time, looking for the closing quote mark. This has the consequence that the literal can't include the closing quote <em>at all</em>, but then backslashes behave completely normally and are easy to understand.</p>
<p>(The above description deliberately glosses over the complication introduced by Python's &quot;triple-quoted&quot; string literals.)</p>
<h2>Making raw strings primary</h2>
<p>Following the Pythonic principle that &quot;explicit is better than implicit&quot;, one could imagine a language in which &quot;raw&quot; strings are the <em>default</em> while strings that use backslash escapes require some special markup (or even an explicit call to an unescaping function, which might be applied at compile-time). This is the approach I am taking in my own design.</p>
<p>With this approach, the standard unescaping scheme would still not be able to use <code>\'</code> or <code>\&quot;</code> escapes (since they could not be present to begin with); but this is not strictly necessary anyway. There are already other commonly understood ways to escape these characters, e.g. with hex codes (<code>\x22</code> for a double quote and <code>\x27</code> for a single quote); or one can invent new ones (perhaps <code>\q</code> for a single quote and <code>\Q</code> for a double quote).</p>
<p>Making the unescaping process explicit also offers the opportunity to specify error handling (should a literal like <code>'\c'</code>, where there is no such escape sequence, produce a string that actually has a backslash in it? Produce a string with only the <code>c</code>? Raise an exception / cause a compilation error? Something else?).</p>
<h2>Adding quotes</h2>
<p>There are several possible workarounds for the inability to include the closing quote mark in such a raw string literal. Being able to choose between <code>'</code> and <code>&quot;</code> delimiters is already helpful, of course. If a string needs to contain both, this can be handled by concatenation with another literal, which could be explicit (using whatever string concatenation operator) or implicit (like <code>&quot;a string with both '&quot; ' and &quot; in it'</code>). Another option is special rules for implicit concatenation - for example, it could be understood that implicit concatenation of two raw literals <em>using the same quote type</em>, like <code>'I' 'll'</code>, implicitly adds an intervening quote of that type (to make <code>I'll</code>). This is, in effect, functionally equivalent to &quot;escaping&quot; that quote character by doubling it up (as seen in some CSV dialects).</p>
<p>In my design, raw literals with the same quote type are implicitly concatenated with that quote when they appear on the same line of the source file, and with a newline when they appear on consecutive lines. The latter makes it easy to write multi-line raw-literal text without needing to worry about leading indentation.</p>

CC BY-SA 4.0
2023-09-11T19:56:06.647Z
2023-09-11T19:56:06.647Z
Karl Knechtel
CC BY-SA 4.0
2023-09-11T20:52:23.183Z
In addition to some CSV dialects, doubling-up the quote character to escape it is also used in [VBScript](https://stackoverflow.com/questions/2942554/adding-quotes-to-a-string-in-vbscript), and apparently [FORTRAN 77](https://docs.oracle.com/cd/E19957-01/805-4939/z40007438613/index.html). I suspect there are several more languages.
kaya3
<p>If the string would generally not contain the closing mark, you actually have the option to not use a special grammar. You could save both the unescaped value and original raw value in the same object, and provide the right value depending on how it is used, say adding a property <code>RawValue</code> to the string type, to get the original string with escaped characters back. In statically typed languages, you could have a separate type to contain strings that are not sure how they will be used, and the raw value could be stripped immediately when you save it into a variable of the type of normal strings, so if the language is compiled, the target code doesn't have to contain both of the versions in most cases.</p>
<p>This is helpful if raw strings are usually expected to be used in some specific scenarios that have known grammar, such as regex.</p>
<p>If you think this is acceptable, you could at least ignore the <code>r&quot;...&quot;</code> style. Other variants are useful for string contents that have unknown format or being formatless, to make closing marks unlikely to appear in the string. But their pros and cons depend on the expected format of the data, which you could only guess.</p>
<p>The Tcl style of nestable matching brackets (actually braces, only counting the not escaped ones, but the escape characters in escaped ones would be kept) probably works best for any formats that is supposed to be human-readable. (Lua is unfortunately not nestable.)</p>
<p>The heredoc style probably works best for copy-pasted data that are possibly ill-formed, but makes noises if the data and code is generated. But it is not that bad, considering that, except for a few cases such as self-extracting scripts, if the data is generated, it is usually better to just refer to an external file. And for self-extracting scripts, you could add a way to read all the data from a marker to the end of file.</p>
<p>I don't think the formats prefixed per line as other answers suggested are actually raw strings. They are escaped by newlines. But if you choose this, you could actually remove the prefix symbols, and just use indentation. While it is not exactly without an escape character, it could be formatted easily using common editors suitable for programming. It should work best in languages like Python, where indentation already have a meaning. It also has the advantage that you could make it automatically remove the indentation in the returned string, so code quoted in code would not become formless.</p>

CC BY-SA 4.0
2024-08-05T22:46:58.173Z
2024-08-08T20:00:33.443Z
2024-08-08T20:00:33.443Z
user23013
<p><strong>PostgreSQL</strong></p>
<ul>
<li>PostgreSQL has the most powerful system I have ever seen for defining different kinds of quotes. In additional to the default single quotes, one can use the double dollar signs with a label in between to create an arbitrary number of new quotation marks. (In a way, this is similar to XML/HTML tags). So, one can include any kinds of quotes inside the string, and is not limited to just single/double/triple quotes/back-ticks.</li>
</ul>
<p>e.g.</p>
<pre class="lang-sql prettyprint-override"><code>SELECT $f$&quot;I'm tired of escaping things like '\'&quot;$f$;
</code></pre>
<pre class="lang-sql prettyprint-override"><code>SELECT $g$&quot;this has a quote $f$ in it &quot;$g$;
</code></pre>
<p>(<code>$f$</code> and <code>$g$</code> are different quotation marks. <code>$g$</code> is effective here as it is on the outside.)</p>
<ul>
<li>Also, PostgreSQL strings are raw by default. You will need to <a href="https://stackoverflow.com/questions/34823158">prepend a <code>E</code>
to the string to use escape sequences</a>.</li>
</ul>
<p>In fact, PostgerSQL string can handle all the examples in the question.</p>
<pre><code>SELECT $f$- Python: `r&quot;I can include \ without worries!&quot;`
- Lua: `[[I can hold arbitrary strings! &quot;'\[] ]]`
- Rust: `r#&quot;String includes quotes? No worries, wrap it in potato shreds&quot;#`$f$
</code></pre>

CC BY-SA 4.0
2024-08-08T20:43:28.593Z
2024-08-08T20:43:28.593Z
tinlyx
CC BY-SA 4.0
2024-08-08T20:48:02.893Z
This seems similar to [OCaml allowing identifiers](https://langdev.stackexchange.com/a/2907/89), although OCaml apparently also allows defining behaviour for string literals based on the identifier used.
kaya3
<p>Some languages that don't have a explicit syntax still have ways of creating raw strings in other ways.</p>
<p>Depending on your language it may not be worth supporting raw strings if you already have something like this:</p>
<pre class="lang-php prettyprint-override"><code>&lt;?php
ob_start();
?&gt;your string that may contains &quot; quotes and \ backslashes&lt;?php
$string = ob_get_clean();
</code></pre>
<p>This technique is commonly used when needing to embed other languages like CSS and Javascript in PHP.</p>
<p>If you have a templating language that already has a builtin for capturing output raw strings may be unnecessary.</p>

CC BY-SA 4.0
2023-06-28T05:41:35.120Z
2023-06-28T05:41:35.120Z
mousetail 'he-him'
<h3>No end in sight</h3>
<p><em>As used by <a href="https://ziglang.org/documentation/master/#Multiline-String-Literals" rel="noreferrer">Zig</a>.</em></p>
<p>Many (all?) of the raw-string formats adopted by mainstream languages still keep to the form start/end delimiter. This evidently immediately creates the issue that the end delimiter must not appear in the string -- else the end is erroneously inferred -- and thereby limits the ability to copy/paste arbitrary segments of text -- especially segments of <em>code</em> -- in the raw-string. Or should I say, make it painful.</p>
<p>We can compare the virtues of <code>{id|</code>..<code>|id}</code> vs <code>r##&quot;</code>..<code>&quot;##</code> vs <code>R&quot;id(</code>..<code>)id&quot;</code> but in the end they all suffer from the same drawback, and neither is that much more readable than the other. They're also hell to tokenize.</p>
<p>Yet, at the same time, almost all mainstream languages already have syntax support for an implicit end delimiter: comment-until-end-of-line.</p>
<p>As such, after many years of mulling this over and over, I would argue that a syntax with &quot;no end marker&quot; should be considered:</p>
<pre><code>let paragraph = #&quot;In Rust, a raw-string is delimited by `r&quot;...&quot;`,
    #&quot;and a matching number of # signs can be added before the opening
    #&quot;quote and after the closing one, in case quotes appears in the
    #&quot;string.
    #&quot;
    ;
</code></pre>
<p>The raw string, here, is composed of everything following the <code>#&quot;</code> sequence, including newlines, with the exception of the last newline.</p>
<p><strong>Advantages</strong>:</p>
<ul>
<li>Multi-lines support.</li>
<li>Nested indentation support, without meaningful indentation necessary.</li>
<li>Consistency: always the same starting delimiter, no arbitrary ID/number of pound signs/... to avoid the token sequence in the string itself.</li>
<li>Ease of copy/paste: paste, indent all lines, prefix all lines. Even in a simple text editor, as long as vertical cursors are supported it's a simple operation, and IDEs support block comments already, which are very similar.</li>
<li>Ease of tokenizing (1): No substring search necessary =&gt; SIMD tokenizers possible.</li>
<li>Ease of tokenizing (2): Parallel tokenization of files, chunked arbitrarily, is possible.</li>
</ul>
<p><strong>Disadvantage</strong>:</p>
<ul>
<li>Vertical Space.</li>
</ul>
<p>Illustration of Vertical Space issue:</p>
<pre class="lang-rust prettyprint-override"><code>//  Rust
string.push_str(r#&quot;&quot;Hello, world&quot;&quot;#);

//  Hypothetical
string.push_str(#&quot;&quot;Hello, world&quot;
    );
</code></pre>
<p>The matter, obviously, compounds if pushing many fragments of raw strings.</p>
<p><strong>Choices</strong>:</p>
<ul>
<li>Interpolation vs Copy/Paste: Interpolation requires treating certain characters specially, so a choice between Interpolation vs Verbatim text must be made. Or no choice, and having two start sequences depending on the case. It's an orthogonal issue.</li>
</ul>

CC BY-SA 4.0
2023-09-12T15:00:23.727Z
2023-09-22T07:15:25.553Z
2023-09-22T07:15:25.553Z
Matthieu M.
CC BY-SA 4.0
2023-09-12T16:39:32.430Z
Another disadvantage: Different things that may or may not count as newlines (e.g., raw carriage returns, form feeds, or vertical tabs might be interpreted differently by the compiler vs. the program rendering the code, creating ambiguity)
rydwolf
CC BY-SA 4.0
2023-09-12T17:15:59.823Z
@RydwolfPrograms: Interesting. Seems akin to the bidirectional Unicode issue. I need to think more about it... though at the same time, wouldn't the issue also plague multi-line comments anyway?
Matthieu M.
CC BY-SA 4.0
2023-09-12T18:26:44.103Z
Most languages need to be able to detect end of line for comments anyways so detecting line breaks is a non-issue. Just treat them the same as anything else that requires newlines
mousetail 'he-him'
CC BY-SA 4.0
2023-09-12T19:16:20.497Z
@mousetail Yeah but in typical code, you'd only ever be using `\n` or `\r\n` ot whatever the standard newline ending is. But raw strings are intended to be potentially used with arbitrary unicode; so it's very possible they would contain form feeds and raw carriage returns and stuff, unlike plain code.
rydwolf
CC BY-SA 4.0
2023-09-13T06:36:18.717Z
@RydwolfPrograms They might effect the visual look but not hurt parsing
mousetail 'he-him'
CC BY-SA 4.0
2023-09-13T13:18:54.760Z
@mousetail The visual look is my whole point. My point is that if the IDE's line ending characters don't match the parsers, that will cause pretty annoying bugs (since you have to figure out where in the string the rogue form feed or whatever is, and add a `#"` before it)
rydwolf
CC BY-SA 4.0
2023-09-14T08:09:26.927Z
@KarlKnechtel: The example I give is exactly passing to a function...
Matthieu M.
<h2>OCaml</h2>
<p>OCaml supports raw string string literals in addition to classic double-quoted strings with escapes:</p>
<ul>
<li>The <a href="https://v2.ocaml.org/manual/lex.html#sss:stringliterals" rel="noreferrer">OCaml Manual</a> has the details.</li>
<li>A raw string is enclosed by <code>{id|</code> and <code>|id}</code> where <code>id</code> is an optional character sequence that must match at the opening and closing delimiter. Most code uses simply <code>{|...|}</code> for raw strings (which now can't contain <code>|}</code>).</li>
<li>The <code>id</code> can be used by the macro (PPX) system to give special meaning to particular strings. You could imagine using <code>{sql|</code>...<code>|sql}</code> for example.</li>
</ul>

CC BY-SA 4.0
2023-09-12T08:12:02.997Z
2023-09-12T08:18:44.513Z
2023-09-12T08:18:44.513Z
Christian Lindig
# What are some different approaches to raw string syntax, and what are their pros and cons?
Posted by: username_0 on 2023-06-28
Many languages include a "raw string" literal type, intended for arbitrary (or as close as possible to arbitrary) strings of characters (and/or bytes), ignoring escape sequences and the like. This can be handy for things like regular expressions, where using typical string literals would require unreadable and bug-inviting double escaping (e.g., `"a\\?\\\\b"` for a regex matching `a?\b`).

I've seen a ton of variations in raw string syntax, with varying capabilities and readability:

* JavaScript: `String.raw`"I'm tired of escaping things like '\'"``
* Python: `r"I can include \ without worries!"`
* Lua: `[[I can hold arbitrary strings! "'\[] ]]`
* Rust: `r#"String includes quotes? No worries, wrap it in potato shreds"#`

What are the advantages of these options over one another, or other alternatives?
**Tags:** syntax strings
> **Comments:**
> **username_1**: Does Rust also allow `##"multiple hashes"##` like in Swift?  
> **username_0**: @Bbrk24 yes, any number  
> **username_2**: Does putting your raw string in a separate file and calling `file_get_contents()` count? Not sure whether that's worth adding as an answer.  
> **username_0**: @TRiG Unless it was a feature built into the language speciifcally for this purpose, I don't think it'd really be an on-topic answer  
> **username_3**: Some relevant discussion here: https://github.com/rust-lang/rust/issues/9411  
> **username_4**: And one type that everyone's missed because it's prehistoric (and not very useful): Hollerith constants. Dating from early versions of Fortran (early 1960s), here's an example of a raw string: `22HThe next 22 characters`.  
> **username_0**: @k314159 You could make that an answer  
---
## 11 answers
### Answer by username_12 on 2023-09-12. Score: 13
No end in sight
As used by Zig.
Many (all?) of the raw-string formats adopted by mainstream languages still keep to the form start/end delimiter. This evidently immediately creates the issue that the end delimiter must not appear in the string -- else the end is erroneously inferred -- and thereby limits the ability to copy/paste arbitrary segments of text -- especially segments of code -- in the raw-string. Or should I say, make it painful.
We can compare the virtues of {id|..|id} vs r##".."## vs R"id(..)id" but in the end they all suffer from the same drawback, and neither is that much more readable than the other. They're also hell to tokenize.
Yet, at the same time, almost all mainstream languages already have syntax support for an implicit end delimiter: comment-until-end-of-line.
As such, after many years of mulling this over and over, I would argue that a syntax with "no end marker" should be considered:
let paragraph = #"In Rust, a raw-string is delimited by `r"..."`,
    #"and a matching number of # signs can be added before the opening
    #"quote and after the closing one, in case quotes appears in the
    #"string.
    #"
    ;

The raw string, here, is composed of everything following the #" sequence, including newlines, with the exception of the last newline.
Advantages:

Multi-lines support.
Nested indentation support, without meaningful indentation necessary.
Consistency: always the same starting delimiter, no arbitrary ID/number of pound signs/... to avoid the token sequence in the string itself.
Ease of copy/paste: paste, indent all lines, prefix all lines. Even in a simple text editor, as long as vertical cursors are supported it's a simple operation, and IDEs support block comments already, which are very similar.
Ease of tokenizing (1): No substring search necessary => SIMD tokenizers possible.
Ease of tokenizing (2): Parallel tokenization of files, chunked arbitrarily, is possible.

Disadvantage:

Vertical Space.

Illustration of Vertical Space issue:
//  Rust
string.push_str(r#""Hello, world""#);

//  Hypothetical
string.push_str(#""Hello, world"
    );

The matter, obviously, compounds if pushing many fragments of raw strings.
Choices:

Interpolation vs Copy/Paste: Interpolation requires treating certain characters specially, so a choice between Interpolation vs Verbatim text must be made. Or no choice, and having two start sequences depending on the case. It's an orthogonal issue.
> **Comments:**
> **username_0** (2023-09-12): Another disadvantage: Different things that may or may not count as newlines (e.g., raw carriage returns, form feeds, or vertical tabs might be interpreted differently by the compiler vs. the program rendering the code, creating ambiguity)  
> **username_12** (2023-09-12): @RydwolfPrograms: Interesting. Seems akin to the bidirectional Unicode issue. I need to think more about it... though at the same time, wouldn't the issue also plague multi-line comments anyway?  
> **username_19** (2023-09-12): Most languages need to be able to detect end of line for comments anyways so detecting line breaks is a non-issue. Just treat them the same as anything else that requires newlines  
> **username_0** (2023-09-12): @mousetail Yeah but in typical code, you'd only ever be using `\n` or `\r\n` ot whatever the standard newline ending is. But raw strings are intended to be potentially used with arbitrary unicode; so it's very possible they would contain form feeds and raw carriage returns and stuff, unlike plain code.  
> **username_19** (2023-09-13): @RydwolfPrograms They might effect the visual look but not hurt parsing  
> **username_0** (2023-09-13): @mousetail The visual look is my whole point. My point is that if the IDE's line ending characters don't match the parsers, that will cause pretty annoying bugs (since you have to figure out where in the string the rogue form feed or whatever is, and add a `#"` before it)  
> **username_12** (2023-09-14): @KarlKnechtel: The example I give is exactly passing to a function...  
---
### Answer by username_20 on 2023-09-12. Score: 9
## OCaml

OCaml supports raw string string literals in addition to classic double-quoted strings with escapes:

* The [OCaml Manual](https://v2.ocaml.org/manual/lex.html#sss:stringliterals) has the details.
* A raw string is enclosed by `{id|` and `|id}` where `id` is an optional character sequence that must match at the opening and closing delimiter. Most code uses simply `{|...|}` for raw strings (which now can't contain `|}`).
* The `id` can be used by the macro (PPX) system to give special meaning to particular strings. You could imagine using `{sql|`...`|sql}` for example.
---
### Answer by username_13 on 2023-06-28. Score: 8
Repeat Quote
Repeat quote three times or more as STRING_QUOTE_TOKEN until the same token is encountered.
Should report a parse error if not found.
Similar to ` in markdown
'''raw string'''

""""
multiline * 4
raw string
""""

The disadvantage is that it is a bit long, at least 3 characters.
The advantage is that you don't need to consider any escaping, you can even dynamically interprete escape.
For simplicity, you can also specify that 'raw' is the raw string '''raw'literal''' is used to deal with the occurrence of ' in the raw string.
The fatal problem is what to do if your first character is ', at this point you have to choose another character like " to form.

The dilemma with escaping is that the more complex your escape, the more complex the unescaped form used to express that character.
If you have written tmLanguage, you may have encountered the situation where you have to write \\\\\\\\.
Therefore, the best way is not to escape, and directly capture the content between the two tokens as a raw string.
> **Comments:**
> **username_14** (2023-09-17): "you don't need to consider escaping" tell that a code generator for the language. I remember having to write ""${""}" in Scala code generation for Scala code to generate raw strings. But it's less of an issue.  
> **username_15** (2024-08-05): My language Papyri does this. The "fatal problem" is not just when you want to start a string with a quote, but also when you want to end with one. I am considering delimiters like `~'''` and `'''~` where the `~` symbol is optional on each side independently, and strips whitespace from that side; this would also mean you can use whitespace for alignment purposes at the start and end of multi-line strings.  
---
### Answer by username_5 on 2023-09-16. Score: 7
I see no mention of Zig so far, so I think it's worth mentioning it as another point in the space.
Zig has regular inline strings with escape sequences like most languages, and multiline strings which have no escapes and require each line to be prefixed with \\.
const hello_world_in_c =
    \\#include 
    \\
    \\int main(int argc, char **argv) {
    \\    printf("hello world\n");
    \\    return 0;
    \\}
;

Although this means that you cannot paste in a block of text directly as a "raw string" without prefixing the lines, it has some advantages:

The grammar remains context-free, unlike Swift, Rust, and others which require matching but arbitrary open/close pairs.
It's impossible to forget to close the string, so in theory this means less catastrophic parse/highlighting failures in an IDE with incremental parsing of broken code.
As Alex Kladov notes, it's possible to lex the file looking at chunks in parallel or incrementally, since the tokens on one line can never affect the tokens on another line.
---
### Answer by username_1 on 2023-09-12. Score: 6
Swift: Raw strings with interpolation
Swift, like Rust, uses hash signs for raw strings. However, you can also interpolate into raw strings by adding the hashes to the interpolation marker:
"Regular string interpolation \(x)"
#"Raw string with "quotes" in it"#
#"Raw string with interpolation \#(x)"#
##"Not an interpolation \#(x)"##
##"But this is \##(x)"##

You can have arbitrarily many hash signs in the literal, but the opening and closing quote marks must match.
> **Comments:**
> **username_6** (2024-08-10): C# also went this route for its design. I love the way Mads Torgersen put phrased it [in this talk](https://youtu.be/v8bqAm4aUFM&t=3170): "You can always win. It might not look nice by the time you win, ... [but you can always win]" You can take any arbitrary string (including other Swift/Rust/C# source code that uses this feature), paste it in directly, and change only the delimiters to make it correct. You don't have to edit the content to appease the host language, and that's awesome.  
---
### Answer by username_7 on 2023-06-28. Score: 5
Go style: backticks only
Similar to JavaScript, Go uses a pair of backticks for raw strings:
"regular"
`raw`

"regular\n    indented\newlines"
`raw
    indented
newlines`

Which is exceptionally useful for regexes, as you don't need to escape anything, except for the usual escaping of special regex characters:
regexp.MustCompile(`(\w)(\w+)(\w)`)

Pros

Extremely simple, no new parsing required (just an additional token kind)
Can be used anywhere a regular string literal is used
You cannot escape anything in these, so \n is the literal characters \ and n.

Cons

You cannot escape anything in these. This leads to backticks being the only disallowed character because it would end the literal, and you can't escape it because you can't escape a raw string. This can be worked around by concatenating with a regular string literal:
query := `SELECT * FROM` + "`table`"


You need to keep track of indention level relative to the start of the raw string for indented parts to have the correct indentation.
> **Comments:**
> **username_8** (2024-08-06): The cons can be resolved as follows: allow more than one backtick, and require the end delimiter to match the begin delimiter.  
> **username_9** (2024-08-08): @ThomasMueller This still has the problem that the string cannot start with a backtick.  
> **username_8** (2024-08-09): Right. That would require using a multi-line raw string: the first line contains two backticks (as the start delimiter), then the second line would contain one backtick, and the last line two backticks (as the end delimiter).  
---
### Answer by username_10 on 2023-06-28. Score: 4
Python style
Pros: Low friction syntax-wise, close enough to existing string syntax
Cons: being closed by a regular quotechar still requires you to include a \ for including a quote value
JavaScript style
Probably the most confusing of all the syntaxes, a very special case in parsing, requires you to recognize String.raw`str` as a function call.
Lua style
Possible confusion with array syntax in other languages  (although in lua's case Arrays are defined by {} so not in this particular case)
Rust style
Maybe the most robust of all syntaxes, it's far enough from other rust syntaxes to be confusing or to be accidentally closed by another value, can't think of a downside
C++
I'd like to add another case of raw strings: C++. It allows the user to define their own syntax for their specific use case, for example
auto rawstr = R"rawstring(
text
text
text
)rawstring";

auto rawstr2 = R"aaaaa(
text
text
text
aaaaa)";

are both raw strings, and the user can choose a termination value that does not conflict with their expected values and for which the start/end of the string is clearly readable.
> **Comments:**
> **username_11** (2023-06-28): I used the C++-like approach in my programming language: https://flatassembler.github.io/AEC_specification.html#multiline_strings  
> **username_12** (2023-09-12): For those not familiar with the given languages, it would be nice to have an example, each time.  
> **username_10** (2023-09-12): @MatthieuM. I only added examples for C++ because it's the only one that the original answer didn't have (as they have examples for the ones they do know about)  
---
### Answer by username_16 on 2023-09-11. Score: 3
## Recognizing the delimiter

The examples given in the question obscure an important mechanical detail relevant to Python's implementation (and possibly others). A Python raw string literal *uses the same lexing/tokenization rule* as other string literals: a string consists of sequences of (anything except the closing quote) *or* (a backslash followed by a single character). It only has different rules for *interpreting* that token (everything is treated literally, including backslashes).

This has two important consequences:

1. A Python raw string literal *can* including the closing quote character, *but only immediately after* a backslash (which the resulting string will contain literally).
2. A Python raw string literal *cannot* end with *an odd number of* backslash characters.

An alternative rule (apparently used in Go, as seen in bigyihsuan's answer) is to just lex the string a character at a time, looking for the closing quote mark. This has the consequence that the literal can't include the closing quote *at all*, but then backslashes behave completely normally and are easy to understand.

(The above description deliberately glosses over the complication introduced by Python's "triple-quoted" string literals.)

## Making raw strings primary

Following the Pythonic principle that "explicit is better than implicit", one could imagine a language in which "raw" strings are the *default* while strings that use backslash escapes require some special markup (or even an explicit call to an unescaping function, which might be applied at compile-time). This is the approach I am taking in my own design.

With this approach, the standard unescaping scheme would still not be able to use `\'` or `\"` escapes (since they could not be present to begin with); but this is not strictly necessary anyway. There are already other commonly understood ways to escape these characters, e.g. with hex codes (`\x22` for a double quote and `\x27` for a single quote); or one can invent new ones (perhaps `\q` for a single quote and `\Q` for a double quote).

Making the unescaping process explicit also offers the opportunity to specify error handling (should a literal like `'\c'`, where there is no such escape sequence, produce a string that actually has a backslash in it? Produce a string with only the `c`? Raise an exception / cause a compilation error? Something else?).

## Adding quotes

There are several possible workarounds for the inability to include the closing quote mark in such a raw string literal. Being able to choose between `'` and `"` delimiters is already helpful, of course. If a string needs to contain both, this can be handled by concatenation with another literal, which could be explicit (using whatever string concatenation operator) or implicit (like `"a string with both '" ' and " in it'`). Another option is special rules for implicit concatenation - for example, it could be understood that implicit concatenation of two raw literals *using the same quote type*, like `'I' 'll'`, implicitly adds an intervening quote of that type (to make `I'll`). This is, in effect, functionally equivalent to "escaping" that quote character by doubling it up (as seen in some CSV dialects).

In my design, raw literals with the same quote type are implicitly concatenated with that quote when they appear on the same line of the source file, and with a newline when they appear on consecutive lines. The latter makes it easy to write multi-line raw-literal text without needing to worry about leading indentation.
> **Comments:**
> **username_15** (2023-09-11): In addition to some CSV dialects, doubling-up the quote character to escape it is also used in [VBScript](https://stackoverflow.com/questions/2942554/adding-quotes-to-a-string-in-vbscript), and apparently [FORTRAN 77](https://docs.oracle.com/cd/E19957-01/805-4939/z40007438613/index.html). I suspect there are several more languages.  
---
### Answer by username_19 on 2023-06-28. Score: 3
Some languages that don't have a explicit syntax still have ways of creating raw strings in other ways.
Depending on your language it may not be worth supporting raw strings if you already have something like this:
your string that may contains " quotes and \ backslashes
This technique is commonly used when needing to embed other languages like CSS and Javascript in PHP.
If you have a templating language that already has a builtin for capturing output raw strings may be unnecessary.
---
### Answer by username_17 on 2024-08-05. Score: 2
If the string would generally not contain the closing mark, you actually have the option to not use a special grammar. You could save both the unescaped value and original raw value in the same object, and provide the right value depending on how it is used, say adding a property `RawValue` to the string type, to get the original string with escaped characters back. In statically typed languages, you could have a separate type to contain strings that are not sure how they will be used, and the raw value could be stripped immediately when you save it into a variable of the type of normal strings, so if the language is compiled, the target code doesn't have to contain both of the versions in most cases.

This is helpful if raw strings are usually expected to be used in some specific scenarios that have known grammar, such as regex.

If you think this is acceptable, you could at least ignore the `r"..."` style. Other variants are useful for string contents that have unknown format or being formatless, to make closing marks unlikely to appear in the string. But their pros and cons depend on the expected format of the data, which you could only guess.

The Tcl style of nestable matching brackets (actually braces, only counting the not escaped ones, but the escape characters in escaped ones would be kept) probably works best for any formats that is supposed to be human-readable. (Lua is unfortunately not nestable.)

The heredoc style probably works best for copy-pasted data that are possibly ill-formed, but makes noises if the data and code is generated. But it is not that bad, considering that, except for a few cases such as self-extracting scripts, if the data is generated, it is usually better to just refer to an external file. And for self-extracting scripts, you could add a way to read all the data from a marker to the end of file.

I don't think the formats prefixed per line as other answers suggested are actually raw strings. They are escaped by newlines. But if you choose this, you could actually remove the prefix symbols, and just use indentation. While it is not exactly without an escape character, it could be formatted easily using common editors suitable for programming. It should work best in languages like Python, where indentation already have a meaning. It also has the advantage that you could make it automatically remove the indentation in the returned string, so code quoted in code would not become formless.
---
### Answer by username_18 on 2024-08-08. Score: 1
PostgreSQL

PostgreSQL has the most powerful system I have ever seen for defining different kinds of quotes. In additional to the default single quotes, one can use the double dollar signs with a label in between to create an arbitrary number of new quotation marks. (In a way, this is similar to XML/HTML tags). So, one can include any kinds of quotes inside the string, and is not limited to just single/double/triple quotes/back-ticks.

e.g.
SELECT $f$"I'm tired of escaping things like '\'"$f$;

SELECT $g$"this has a quote $f$ in it "$g$;

($f$ and $g$ are different quotation marks. $g$ is effective here as it is on the outside.)

Also, PostgreSQL strings are raw by default. You will need to prepend a E
to the string to use escape sequences.

In fact, PostgerSQL string can handle all the examples in the question.
SELECT $f$- Python: `r"I can include \ without worries!"`
- Lua: `[[I can hold arbitrary strings! "'\[] ]]`
- Rust: `r#"String includes quotes? No worries, wrap it in potato shreds"#`$f$
> **Comments:**
> **username_15** (2024-08-08): This seems similar to [OCaml allowing identifiers](https://langdev.stackexchange.com/a/2907/89), although OCaml apparently also allows defining behaviour for string literals based on the identifier used.  
---
