<p>New users of a language will spend a lot of their time staring at error messages, and realistically so will the language designers. These messages should be an early consideration, given that adding new information might require extensive compiler changes, and good error messages save time.</p>
<p>The range of message quality seems very wide, from the <a href="https://codegolf.stackexchange.com/questions/1956/generate-the-longest-error-message-in-c/10470#10470">infamous C++ errors</a> (here shown deliberately cryptic):</p>
<p><a href="https://i.sstatic.net/w9zJl.png" rel="noreferrer"><img src="https://i.sstatic.net/w9zJl.png" alt="Crop from a long C++ error message" /></a></p>
<p>to Nushell's eye candy:</p>
<p><a href="https://www.nushell.sh/#nu-has-great-error-messages" rel="noreferrer"><img src="https://i.sstatic.net/ic1pM.png" alt="Error message in the Nu shell programming language showing various colors, text formatting, and Unicode arrows." /></a></p>
<p>Given the effort that even <a href="https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-pep657" rel="noreferrer">mainstream</a> <a href="https://doc.rust-lang.org/error_codes/error-index.html" rel="noreferrer">languages</a> are spending on this topic, and the warm reception to these improvements, <strong>what are the best practices for displaying error messages</strong>?</p>
<hr />
<p><em>Note that this question is about the user interface of the compiler. Suggesting &quot;report as many errors as possible&quot; is ok, but specifics on <a href="https://langdev.stackexchange.com/questions/189/what-are-some-ways-of-reporting-compilation-errors">how to implement such a compiler</a> are out-of-scope. Syntax and runtime error messages can be included if there are special considerations.</em></p>

<error-handling><language-ergonomics>
What should be included in error messages?
CC BY-SA 4.0
2023-06-30T13:20:02.720Z
Related: [Are very explanatory compiler error messages worth their weight in work needed to support them?](https://langdev.stackexchange.com/q/544/156)
pppery
CC BY-SA 4.0
2023-07-02T13:40:14.943Z
This is our first question to make the HNQ!
Starship
CC BY-SA 4.0
2023-07-02T16:56:10.123Z
Which one? @CaveJohnson
Starship
CC BY-SA 4.0
2023-07-02T16:56:50.137Z
Do you mean this?https://langdev.stackexchange.com/questions/1815/how-could-a-language-make-the-loop-and-a-half-less-error-prone @CaveJohnson
Starship
CC BY-SA 4.0
2023-07-02T17:00:18.043Z
Okay then sure @CaveJohnson
Starship
<h2>Unique Error Code</h2>
<p>Often overlooked, but very necessary for non-English speakers like me.</p>
<p>Many problems have been solved in detail in mainstream languages, but localized errors are probably not solved in localized search engines.</p>
<p>And with a unified error code, you can use <code>Rust &quot;E0308&quot;</code> to search for the cause of the error and the solution in other languages.</p>
<p>The error code also facilitates downgrading to C language FFI.</p>
<h2>Where the Error Occurred</h2>
<p>It is best to be a hyperlink, click to jump to the location where the error occurred, which is convenient for debugging.</p>
<p>The format is generally</p>
<p><code>mopno_project/sub_project/module/code.c:line:col</code></p>
<p>Note that <strong>whenever a path appears, it should be relative to the project root</strong>.</p>
<p>Otherwise, it is very easy to leak privacy or secrets.</p>
<h2>When the Error Occurred</h2>
<p>If it is a runtime error, show the event when the error occurred, using a human-friendly output form, such as RFC3339.</p>
<h2>Why the Error Occurred and How to Fix</h2>
<p>The main cause of the error needs to be given, so that the programmer can quickly understand the problem.</p>
<p>If possible, give possible fixes, or templates for fixes.</p>
<p>If it is an html environment like jupyter, it can support copying by click.</p>
<p>Simple internal errors IDE will usually prompt you to use <a href="https://www.jetbrains.com/help/idea/resolving-problems.html" rel="noreferrer">quick-fix</a>.</p>
<p>But if it is an external error, the IDE generally does not support automatic repair. For example, if there is a Regex syntax problem, then the library author needs to provide a repair solution.</p>
<p>This method is not as convenient as <code>quick-fix</code>, but at least I haven't encountered a language that can use libraries to define IDE behavior.</p>
<h2>Stack Backtrace</h2>
<p>In debug mode a stack trace should be generated, printing each step of the call.</p>
<p>I think actually <code>print</code> should also generate a stack trace in debug mode.</p>
<p>But the overhead is huge, so don't use it as the default behavior, and don't print it in release unless specified by the user.</p>

CC BY-SA 4.0
2023-06-30T12:48:37.620Z
2023-06-30T13:34:06.223Z
2023-06-30T13:34:06.223Z
Aster
CC BY-SA 4.0
2023-07-01T02:31:08.600Z
The point about fixes is a potentially risky one. For that to actually be useful information, the person writing the error detection needs to be certain about where the error actually is, what the error is, and what the developer actually meant to do. This is easy for syntax errors if you write your parser correctly, but can be very difficult for runtime errors, and if the ‘fix’ is incorrect it’s counterproductive.
Austin Hemmelgarn
CC BY-SA 4.0
2023-07-02T09:48:17.817Z
One thing I'd note there. Once an error code has been allocated it should never, under any circumstances, be changed: doing so would render all archived documentation including textbooks and all web searches obsolete. In principle, even a change to an error should result in the existing code being retired, i.e. "don't expect to see this after version x.y". All of which makes organising them by topic and keeping them short a tricky undertaking,
Mark Morgan Lloyd
<h3>It depends on the language</h3>
<p>When running with a debugger, or the language is interpreted and in interactive mode, less information is necessary upfront. For example, there's no need to dump the stack to screen, or what the source file was, if this information is available upon request. In my opinion, it is better to keep the initial information clean and concise, with just enough information to tell me what went wrong and where. As an example, take APL, which (for over half a century!) has been printing a brief error message, the function name and line number, with the code line, and an indicator as to where on the line the error happened:</p>
<pre class="lang-none prettyprint-override"><code>      foo
DOMAIN ERROR
foo[42] 1÷0
         ∧
</code></pre>
<p>Newer systems add a bit more help in the message:</p>
<pre class="lang-none prettyprint-override"><code>DOMAIN ERROR: Divide by zero
foo[42] 1÷0
         ∧
</code></pre>
<p>Sometimes, it pinpoints which part of the data was bad:</p>
<pre><code>DOMAIN ERROR: JSON export: item &quot;bad[4]&quot; of the right argument (⎕IO=1) cannot be converted
      1 ⎕JSON ns
        ∧
</code></pre>
<p>Or conveys OS errors:</p>
<pre><code>FILE NAME ERROR: /tmp/badfile.txt: Unable to open file (&quot;The system cannot find the file specified.&quot;)
</code></pre>
<p>Either way, the programmer now navigate up and down the stack, and ask for more information about the error from a special &quot;extended diagnostic message&quot; object. This can contain the specific error codes, who is responsible for the erroing code, the source location in the implementation language, and even a URL where the user can get help.</p>

CC BY-SA 4.0
2023-06-30T13:32:07.827Z
2023-06-30T13:32:07.827Z
Adám
<h2>Recognise that source code might trigger a compiler internal error</h2>
<p>The message should clarify if it is an <em>internal error</em> in the compiler, .v.s. an error in the user's source code.</p>
<p>While most errors are expected to be in the user's source code, sometimes the input causes an internal compiler error which needs to be fed back to the compiler writer.</p>
<p>An <em>internal error</em> in a compiler could potentially be caused by invalid or valid source code.</p>
<p>The Texas Instruments compilers give a message to contact customer support if an internal error occurs. Some examples are:</p>
<ol>
<li><a href="https://e2e.ti.com/support/tools/code-composer-studio-group/ccs/f/code-composer-studio-forum/527777/incorrect-use-of-_symval-causes-ti-msp430-v15-12-2-2-lts-internal-compiler-error" rel="nofollow noreferrer">Incorrect use of _symval causes TI MSP430 v15.12.2.2.LTS internal compiler error</a> which was caused by an error in the source code.</li>
<li><a href="https://e2e.ti.com/support/tools/code-composer-studio-group/ccs/f/code-composer-studio-forum/822435/compiler-tms320f28377d" rel="nofollow noreferrer">Compiler/TMS320F28377D</a> which looking at the compiler bug report <a href="https://sir.ext.ti.com/jira/browse/EXT_EP-9417" rel="nofollow noreferrer">Using --vcu_support may lead to INTERNAL ERROR: Register allocation failed</a> was caused by valid source code.</li>
</ol>
<p>Another example is AdaCore documenting in <a href="https://docs.adacore.com/gnat_ugn-docs/html/gnat_ugn/gnat_ugn/gnat_and_program_execution.html#gnat-abnormal-termination-or-failure-to-terminate" rel="nofollow noreferrer">GNAT Abnormal Termination or Failure to Terminate</a> how a user can investigate the language construct which triggers an internal error in the compiler.</p>

CC BY-SA 4.0
2023-07-01T19:17:31.583Z
2023-07-01T19:17:31.583Z
Chester Gillon
<p>To complement Aster's answer:</p>
<h2>Include code location excerpt</h2>
<p>Saying <code>error_code.cpp:8:89</code> is useful for tools and links, but the user shouldn't have to search for every code mention. Include an excerpt of the code location, preferably with a bit of context.</p>
<p>Rust example:</p>
<pre class="lang-rust prettyprint-override"><code> --&gt; src/main.rs:4:48
  |
3 | fn main() {
4 |     let greeting_file = File::open(&quot;hello.txt&quot;)?;
  |                                                ^
</code></pre>
<p>Note the <code>^</code>, used to highlight the specific location of the error. This is important for long lines, or ambiguous locations.</p>
<p>If possible, include the line where the surrounding function was declared, or at least its name.</p>
<p>Colors are nice, but make sure that the output is understandable without them.</p>
<h2>A flag to report errors as structured data</h2>
<p>The compiler output might be consumed by automated tools and <a href="https://langdev.stackexchange.com/questions/1558/how-can-a-compilers-parse-errors-be-tested">tests</a>, so it's important to give them an output format that is parsable and stable.</p>
<p>See Rust's <a href="https://doc.rust-lang.org/rustc/command-line-arguments.html#option-error-format" rel="noreferrer"><code>-error-format={human,json,short}</code></a> CLI option.</p>
<h2>List as many errors as possible</h2>
<p>It's tempting to stop parsing/compilation when the first error is found, but this leads to a frustrating dev experience. Include as many errors as possible in the report, taking care to avoid duplication.</p>
<h2>Help typos by suggesting similar names</h2>
<p>If the compilation error is that <code>variable &quot;user_pos&quot; is not defined</code>, and there's a variable <code>userpos</code> in that scope, that's worth mentioning in the error message. <a href="https://en.wikipedia.org/wiki/Levenshtein_distance" rel="noreferrer">Levenshtein distance</a> provides an easy-to-compute way of finding the similarity of two strings, be they variables, functions, or attributes.</p>

CC BY-SA 4.0
2023-06-30T16:57:17.127Z
2023-06-30T17:38:55.880Z
2023-06-30T17:38:55.880Z
BoppreH
<p>As someone who has to read a lot of errors, the most useful words are a severity indicator in one line.</p>
<ul>
<li>DEBUG: ....details....</li>
<li>INFO: blah</li>
<li>WARNING: blah</li>
<li>ERROR: but continuing</li>
<li>ERROR: but continuing (but with significant component disabled)</li>
<li>FATAL ERROR: terminating (and why)</li>
</ul>
<p>Don't confuse an error message with a backtrace - they're different because an error should show succinctly what's wrong, and the backtrace is more &quot;how we got to this point&quot;</p>
<p>(aside) Yes these severity levels map onto syslog's hierarchy tree.  That's a well defined and understood scale which can be applied to any importance-ranking.</p>
<p>(aside2) Why should all the important info one line?   Log analyser software has an easier job without having to keep state of multiple logging lines/output.</p>

CC BY-SA 4.0
2023-07-01T01:14:32.997Z
2023-07-01T22:05:31.723Z
2023-07-01T22:05:31.723Z
Criggie
CC BY-SA 4.0
2023-07-01T01:16:41.347Z
How often are compiler errors going into a syslog rather than being read immediately on the console output, though?
kaya3
CC BY-SA 4.0
2023-07-01T01:26:14.263Z
@kaya3-supportthestrike the syslog reference was just about the well-known relative scale of "debug-info-warn-error"... etc I didn't mean that running syslog itself was part of the answer.
Criggie
CC BY-SA 4.0
2023-07-01T03:17:46.923Z
@kaya3-supportthestrike I don't think this question is specific to compilers, and for interpreted languages, you could easily have errors popping up in something running in the background logging to something other than a console
rydwolf
CC BY-SA 4.0
2023-07-01T13:42:37.620Z
@RydwolfPrograms The question says: *"Note that this question is about the user interface of the compiler."* That said, you have a point, but I still think this answer needs to be clarified, since the part about log analyser software is probably not relevant to compilers.
kaya3
# What should be included in error messages?
Posted by: username_0 on 2023-06-30
New users of a language will spend a lot of their time staring at error messages, and realistically so will the language designers. These messages should be an early consideration, given that adding new information might require extensive compiler changes, and good error messages save time.

The range of message quality seems very wide, from the [infamous C++ errors](https://codegolf.stackexchange.com/questions/1956/generate-the-longest-error-message-in-c/10470#10470) (here shown deliberately cryptic):

[![Crop from a long C++ error message](https://i.sstatic.net/w9zJl.png)](https://i.sstatic.net/w9zJl.png)

to Nushell's eye candy:

[![Error message in the Nu shell programming language showing various colors, text formatting, and Unicode arrows.](https://i.sstatic.net/ic1pM.png)](https://www.nushell.sh/#nu-has-great-error-messages)

Given the effort that even [mainstream](https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-pep657) [languages](https://doc.rust-lang.org/error_codes/error-index.html) are spending on this topic, and the warm reception to these improvements, **what are the best practices for displaying error messages**?

---

*Note that this question is about the user interface of the compiler. Suggesting "report as many errors as possible" is ok, but specifics on [how to implement such a compiler](https://langdev.stackexchange.com/questions/189/what-are-some-ways-of-reporting-compilation-errors) are out-of-scope. Syntax and runtime error messages can be included if there are special considerations.*
**Tags:** error-handling language-ergonomics
> **Comments:**
> **username_1**: Related: [Are very explanatory compiler error messages worth their weight in work needed to support them?](https://langdev.stackexchange.com/q/544/156)  
> **username_2**: This is our first question to make the HNQ!  
> **username_2**: Which one? @CaveJohnson  
> **username_2**: Do you mean this?https://langdev.stackexchange.com/questions/1815/how-could-a-language-make-the-loop-and-a-half-less-error-prone @CaveJohnson  
> **username_2**: Okay then sure @CaveJohnson  
---
## 5 answers
### Answer by username_3 on 2023-06-30. Score: 21 | Accepted answer
## Unique Error Code

Often overlooked, but very necessary for non-English speakers like me.

Many problems have been solved in detail in mainstream languages, but localized errors are probably not solved in localized search engines.

And with a unified error code, you can use `Rust "E0308"` to search for the cause of the error and the solution in other languages.

The error code also facilitates downgrading to C language FFI.

## Where the Error Occurred

It is best to be a hyperlink, click to jump to the location where the error occurred, which is convenient for debugging.

The format is generally

`mopno_project/sub_project/module/code.c:line:col`

Note that **whenever a path appears, it should be relative to the project root**.

Otherwise, it is very easy to leak privacy or secrets.

## When the Error Occurred

If it is a runtime error, show the event when the error occurred, using a human-friendly output form, such as RFC3339.

## Why the Error Occurred and How to Fix

The main cause of the error needs to be given, so that the programmer can quickly understand the problem.

If possible, give possible fixes, or templates for fixes.

If it is an html environment like jupyter, it can support copying by click.

Simple internal errors IDE will usually prompt you to use [quick-fix](https://www.jetbrains.com/help/idea/resolving-problems.html).

But if it is an external error, the IDE generally does not support automatic repair. For example, if there is a Regex syntax problem, then the library author needs to provide a repair solution.

This method is not as convenient as `quick-fix`, but at least I haven't encountered a language that can use libraries to define IDE behavior.

## Stack Backtrace

In debug mode a stack trace should be generated, printing each step of the call.

I think actually `print` should also generate a stack trace in debug mode.

But the overhead is huge, so don't use it as the default behavior, and don't print it in release unless specified by the user.
> **Comments:**
> **username_4** (2023-07-01): The point about fixes is a potentially risky one. For that to actually be useful information, the person writing the error detection needs to be certain about where the error actually is, what the error is, and what the developer actually meant to do. This is easy for syntax errors if you write your parser correctly, but can be very difficult for runtime errors, and if the ‘fix’ is incorrect it’s counterproductive.  
> **username_5** (2023-07-02): One thing I'd note there. Once an error code has been allocated it should never, under any circumstances, be changed: doing so would render all archived documentation including textbooks and all web searches obsolete. In principle, even a change to an error should result in the existing code being retired, i.e. "don't expect to see this after version x.y". All of which makes organising them by topic and keeping them short a tricky undertaking,  
---
### Answer by username_0 on 2023-06-30. Score: 8
To complement Aster's answer:
Include code location excerpt
Saying error_code.cpp:8:89 is useful for tools and links, but the user shouldn't have to search for every code mention. Include an excerpt of the code location, preferably with a bit of context.
Rust example:
 --> src/main.rs:4:48
  |
3 | fn main() {
4 |     let greeting_file = File::open("hello.txt")?;
  |                                                ^

Note the ^, used to highlight the specific location of the error. This is important for long lines, or ambiguous locations.
If possible, include the line where the surrounding function was declared, or at least its name.
Colors are nice, but make sure that the output is understandable without them.
A flag to report errors as structured data
The compiler output might be consumed by automated tools and tests, so it's important to give them an output format that is parsable and stable.
See Rust's -error-format={human,json,short} CLI option.
List as many errors as possible
It's tempting to stop parsing/compilation when the first error is found, but this leads to a frustrating dev experience. Include as many errors as possible in the report, taking care to avoid duplication.
Help typos by suggesting similar names
If the compilation error is that variable "user_pos" is not defined, and there's a variable userpos in that scope, that's worth mentioning in the error message. Levenshtein distance provides an easy-to-compute way of finding the similarity of two strings, be they variables, functions, or attributes.
---
### Answer by username_6 on 2023-06-30. Score: 7
It depends on the language
When running with a debugger, or the language is interpreted and in interactive mode, less information is necessary upfront. For example, there's no need to dump the stack to screen, or what the source file was, if this information is available upon request. In my opinion, it is better to keep the initial information clean and concise, with just enough information to tell me what went wrong and where. As an example, take APL, which (for over half a century!) has been printing a brief error message, the function name and line number, with the code line, and an indicator as to where on the line the error happened:
      foo
DOMAIN ERROR
foo[42] 1÷0
         ∧

Newer systems add a bit more help in the message:
DOMAIN ERROR: Divide by zero
foo[42] 1÷0
         ∧

Sometimes, it pinpoints which part of the data was bad:
DOMAIN ERROR: JSON export: item "bad[4]" of the right argument (⎕IO=1) cannot be converted
      1 ⎕JSON ns
        ∧

Or conveys OS errors:
FILE NAME ERROR: /tmp/badfile.txt: Unable to open file ("The system cannot find the file specified.")

Either way, the programmer now navigate up and down the stack, and ask for more information about the error from a special "extended diagnostic message" object. This can contain the specific error codes, who is responsible for the erroing code, the source location in the implementation language, and even a URL where the user can get help.
---
### Answer by username_7 on 2023-07-01. Score: 4
## Recognise that source code might trigger a compiler internal error

The message should clarify if it is an *internal error* in the compiler, .v.s. an error in the user's source code.

While most errors are expected to be in the user's source code, sometimes the input causes an internal compiler error which needs to be fed back to the compiler writer.

An *internal error* in a compiler could potentially be caused by invalid or valid source code.

The Texas Instruments compilers give a message to contact customer support if an internal error occurs. Some examples are:

1. [Incorrect use of \_symval causes TI MSP430 v15.12.2.2.LTS internal compiler error](https://e2e.ti.com/support/tools/code-composer-studio-group/ccs/f/code-composer-studio-forum/527777/incorrect-use-of-_symval-causes-ti-msp430-v15-12-2-2-lts-internal-compiler-error) which was caused by an error in the source code.
2. [Compiler/TMS320F28377D](https://e2e.ti.com/support/tools/code-composer-studio-group/ccs/f/code-composer-studio-forum/822435/compiler-tms320f28377d) which looking at the compiler bug report [Using --vcu\_support may lead to INTERNAL ERROR: Register allocation failed](https://sir.ext.ti.com/jira/browse/EXT_EP-9417) was caused by valid source code.

Another example is AdaCore documenting in [GNAT Abnormal Termination or Failure to Terminate](https://docs.adacore.com/gnat_ugn-docs/html/gnat_ugn/gnat_ugn/gnat_and_program_execution.html#gnat-abnormal-termination-or-failure-to-terminate) how a user can investigate the language construct which triggers an internal error in the compiler.
---
### Answer by username_8 on 2023-07-01. Score: 3
As someone who has to read a lot of errors, the most useful words are a severity indicator in one line.

* DEBUG: ....details....
* INFO: blah
* WARNING: blah
* ERROR: but continuing
* ERROR: but continuing (but with significant component disabled)
* FATAL ERROR: terminating (and why)

Don't confuse an error message with a backtrace - they're different because an error should show succinctly what's wrong, and the backtrace is more "how we got to this point"

(aside) Yes these severity levels map onto syslog's hierarchy tree. That's a well defined and understood scale which can be applied to any importance-ranking.

(aside2) Why should all the important info one line? Log analyser software has an easier job without having to keep state of multiple logging lines/output.
> **Comments:**
> **username_9** (2023-07-01): How often are compiler errors going into a syslog rather than being read immediately on the console output, though?  
> **username_8** (2023-07-01): @kaya3-supportthestrike the syslog reference was just about the well-known relative scale of "debug-info-warn-error"... etc I didn't mean that running syslog itself was part of the answer.  
> **username_10** (2023-07-01): @kaya3-supportthestrike I don't think this question is specific to compilers, and for interpreted languages, you could easily have errors popping up in something running in the background logging to something other than a console  
> **username_9** (2023-07-01): @RydwolfPrograms The question says: *"Note that this question is about the user interface of the compiler."* That said, you have a point, but I still think this answer needs to be clarified, since the part about log analyser software is probably not relevant to compilers.  
---
