<h2>Use case</h2>
<p>Some languages offer techniques to ensure certain requirements at compile time. For example, rust has the <code>NonZeroU32</code> type that will ensure at compile time you can't assign a zero value to it without checking first.</p>
<p>In more complex cases, it can be hard to see if some set of constraints actually restricts the output type as expected. Consider this type in typescript:</p>
<pre><code>type Obj = {... some fields};

const getObj = async&lt;T extends keyof Obj&gt; (id: number, fields: Array&lt;T&gt;): Promise&lt;null | Pick&lt;Obj, T&gt;&gt; =&gt;
            GET(`/api/${ 
                id
            }?fields=${fields.join(',')}`);

// This should fail
getObj(12, [&quot;bla&quot;]).foo
</code></pre>
<p>This is intended to return a type with only the fields passed as strings returned (the rest won't even be fetched from the API). However, the way this works is a bit convoluted and it would be easy to make a mistake that would lead to it allowing accessing other fields, which would crash at runtime. I would want to ensure that can't happen. I can't add test cases for this though because if it failed it would prevent the entire test suit from compiling.</p>
<p>This is just an example, I'm looking for language design features that would allow doing this naturally, not advice on this specific case.</p>
<p>Some other examples of failures I might want to test for:</p>
<pre><code>let x: NonZeroU32 = 0;
</code></pre>
<p>or</p>
<pre><code>println!(&quot;{}&quot;); // bad macro parameters
</code></pre>
<h2>Question</h2>
<p>Do any languages have systems that allow testing code that does not compile? How might such a system be designed?</p>
<p>I'm not looking for advice for this case specifically but more generally if other languages have methods to deal with similar situation.</p>

<static-typing><testing><meta-programming>
"Testing" that something does not compile in metaprogramming
CC BY-SA 4.0
2024-02-05T13:20:44.173Z
Maybe something like duck typing? I feel like it's a similar concept, where while it doesn't match a valid type/matches a disallowed type, the thing still quacks.
Someone
CC BY-SA 4.0
2024-02-05T13:25:54.113Z
@Someone Wouldn't that defeat the whole purpose? The whole reason for defining the types like this is to make sure that it fails at compile time if it's not the right type
mousetail 'he-him'
CC BY-SA 4.0
2024-02-05T13:36:19.377Z
Maybe a strongly-typed language could have a `-B` "broken" option that uses duck typing? I mean, it seems close enough.
Someone
CC BY-SA 4.0
2024-02-05T13:40:31.377Z
@Someone Feel free to add that as an answer. I don't know exactly what you mean but maybe it's what I'm looking for if expanded a bit more
mousetail 'he-him'
CC BY-SA 4.0
2024-02-05T20:39:04.230Z
@Someone if the language has name-based typing resorting to duck typing will influence type equality which will likely break other things too
feldentm
CC BY-SA 4.0
2024-02-05T21:55:31.520Z
The Scala community has longe since raised the more general question: given that our modern type systems are of comparable power to our modern programming languages, what should the tooling look like? What is the type-level equivalent of an IDE, a semantic highlighter, a debugger, a profiler, a test framework, a coverage analyzer? Note that I carefully phrased this as "the Scala community has raised the question", as I believe they have no found a good *answer* yet. But at least, they recognize the problem.
Jörg W Mittag
<blockquote>
<p>Do any languages have systems that allow testing code that does not compile?</p>
</blockquote>
<p>Yes, several languages make it possible to compile code during runtime.</p>
<ul>
<li>Perl has the <code>eval</code> method, which takes a string and compiles it during runtime (implicitly, before running said code). It makes it <a href="https://perldoc.perl.org/functions/eval" rel="nofollow noreferrer">easily possible</a> for the caller to work with any kind of error condition (compilation error or execution error).</li>
<li>Ruby has the <code>Kernel.eval</code> <a href="https://apidock.com/ruby/Kernel/eval" rel="nofollow noreferrer">method</a> which allows for much the same. As everything in Ruby it relies on the usual exception handling, leaving it up to the caller to see if there were compilation errors.</li>
<li>Java can do this too, albeit more involved and less integrated into the base language, by <a href="https://stackoverflow.com/questions/2946338/how-do-i-programmatically-compile-and-instantiate-a-java-class">exposing the Java compiler</a> as user-callable code.</li>
<li>For your own example, <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval" rel="nofollow noreferrer">JavaScript has eval()</a> too. (And for typescript there is a <a href="https://stackoverflow.com/questions/45153848/evaluate-typescript-from-string">transpile()</a> method.)</li>
<li>Even low level languages like C, which most definitely do not include a compiler in their executables, have been using the method of like simply calling the C compiler as a sub-process, on some piece of source code which has been created by the user program dynamically. This is heavily used, in, for example, the <code>automake</code> and similar build systems, to be able to test the compilation environment of some Unix tool and figure out which of a myriad of options said environment offers. This method can arguably used in <em>every</em> language which allows to run sub-processes, and where the compilation environment is available while the tests are running (which might or might not be trivial, if you are thinking about running your tests in an otherwise &quot;lean&quot; CI/CD runner environment, but certainly possible to achieve).</li>
</ul>
<p>Using these tools, it is straightforward to write tests that make sure that the compilation fails, via some pseudo code:</p>
<pre><code>test() {
  got_error = false;
  try {
    eval(&quot;this does not compile!&quot;);
  } catch (CompilationException) {    
    got_error = true;
  }
  assert(got_error);
}
</code></pre>
<p>Adjust by taste.</p>

CC BY-SA 4.0
2024-02-06T10:57:37.890Z
2024-02-06T15:05:25.617Z
2024-02-06T15:05:25.617Z
AnoE
CC BY-SA 4.0
2024-02-06T11:03:01.780Z
Nitpick: You can't `eval` typescript code. You'd need to run the typescript as a sub-process like your C answer
mousetail 'he-him'
CC BY-SA 4.0
2024-02-06T15:06:12.133Z
Nitpick on your nitpick: there seems to be a `transpile()` which seems to be doing exactly that. I have added/clarified it. It's not the most important detail of the answer as OP was adamant in it being a generic question, with TS just an example; anyways. ;)
AnoE
<p>Tyr had noCompile as a test category even in the first published version (<a href="https://github.com/tyr-lang/test.conformance/blob/a7e5a0f44e4cedb5de2cbe54352862dd76e2ed6a/0.4.0/accept/ref/mar.tyr#L10" rel="nofollow noreferrer">example</a>). The question can be split into two aspects.</p>
<p>The first is regular types that exclude certain values. Positive, nonzero and nonnull would be examples for such excluding subtypes. Implementation wise, they behave mostly like their base types except that they need to carry a proof that excluded values cannot reach values of the type. If this is not possible, the implementation can still resort to a runtime check or simply fail. While this category is mentioned explicitly in the question, it is not really about failing compilation. Personally, I would always have such checks as runtime checks because type systems usually lack the expressiveness, i.e. analysis quality, to make such features useful in real world code bases.</p>
<p>The second aspect is actual failing compilation.
Here, one must be aware what not compiling means for the compiler.
When it comes to syntax errors, there might be no reasonable way to recover from that situation and return to the boundaries of the noCompile expression or block or whatever is offered.
When it comes to type errors, the compiler needs thorough tracking of what made it reach that type.
Also, some errors are there to protect the compiler from running into an infinite loop. Getting out of such loop can mean to abort translation in a way that might not be suitable for recovery.
Finally, what might be unexpected, is that evaluation of malformed code can result in the compilers data structures such as the type context getting poisoned with nonsense. Depending on the error class, it can be hard to impossible to allow a compiler to execute the next phase or revert all its changes inside the noCompile context to allow it to produce a usable binary in the end. An example for such an issue is a malformed template type that results in an infinite materialization chain that gets aborted by a limit. This will itself cause an error in a phase requiring all partly materialized types to get fully materialized.</p>
<p>If you think about adding such a feature to your language, have a look at the implementation. If your error category is implemented by just adding an error message to the list and continuing with compilation afterwards, you might likely be fine. If you throw an exception and hope for some surrounding context to abort the member or wherever you can continue, you might exclude the category from such a feature.</p>
<p>TBH, I do not see why one would use a feature like noCompile outside of tests. The primary reason to allow regular programmers to write such tests is to ensure that APIs do not have certain properties like revealing internal state.</p>

CC BY-SA 4.0
2024-02-05T20:35:31.637Z
2024-02-05T20:35:31.637Z
feldentm
CC BY-SA 4.0
2024-02-05T20:44:01.690Z
an example for inaccessible member test can be found here: https://github.com/tyr-lang/test.conformance/blob/a7e5a0f44e4cedb5de2cbe54352862dd76e2ed6a/0.5.0/accept/vcalls/mar.tyr#L18
feldentm
<p>Compile-time errors due to typing are usually in strongly-typed languages. But if a type is too complicated for a compiler, perhaps a runtime type evaluation is needed.</p>
<p>I don't believe this exists in a language as of now (then again, there are <em>thousands</em> of esolangs), but perhaps some kind of compile/runtime flag?</p>
<h1>proposal (probably terrible)</h1>
<p>Essentially, when compiling, there can be a <code>-B</code> (for <strong>b</strong>roken) flag that ignores type errors and switches to a duck typing stance. Essentially, while the types might <em>technically</em> not work, the untyped (or weakly typed) code still might, which is the whole idea behind duck typing.</p>

CC BY-SA 4.0
2024-02-05T13:55:39.610Z
2024-02-05T13:55:39.610Z
Someone
CC BY-SA 4.0
2024-02-05T14:45:59.537Z
How will this let me assert that some construct fails to compile?
mousetail 'he-him'
CC BY-SA 4.0
2024-02-05T15:09:40.580Z
I…think I answered a slightly different question. Testing whether something compiles or not is something different entirely.
Someone
<p>In C++, and possibly other languages, we have the <a href="https://benjaminbrock.net/blog/detection_idiom.php" rel="nofollow noreferrer">Detection Idiom</a>.  This relies on <a href="https://en.cppreference.com/w/cpp/language/sfinae" rel="nofollow noreferrer">SFINAE</a> or Concepts to determine whether template code can be instantiated, yielding a <code>bool</code> which you can <code>static_assert()</code> as a test.</p>
<p>I don't have any examples to show, though I do intend to experiment a bit with this and perhaps create some utility code to make it easier to use.</p>

CC BY-SA 4.0
2024-02-05T16:12:39.087Z
2024-02-05T16:12:39.087Z
Toby Speight
CC BY-SA 4.0
2024-02-10T15:50:51.033Z
Yes, that's the Concepts version of the detection idiom that I'm describing.
Toby Speight
<p><a href="https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-9.html#-ts-expect-error-comments" rel="noreferrer">TypeScript 3.9+ has a feature</a> for this: <code>// @ts-expect-error</code> comments. In an example like yours, it would be:</p>
<pre class="lang-typescript prettyprint-override"><code>type Obj = {bla: string, foo: string};

const getObj = &lt;T extends keyof Obj&gt; (id: number, fields: Array&lt;T&gt;): Pick&lt;Obj, T&gt; =&gt; {
    throw 'empty body';
}

// @ts-expect-error Property 'foo' should not exist
getObj(12, [&quot;bla&quot;]).foo
</code></pre>
<p><a href="https://www.typescriptlang.org/play?#code/C4TwDgpgBA8gRgKygXigbzgGwIYC4oDOwATgJYB2A5gDRQBmA9g-kWVQL4DcAUNwMYNyRKJQjB4SVAB4AKlAgAPYBHIATAlADWEEAzqxEAPigAKUqvzkArgFs4EYrTqkImdfgCCxYthCzDAJT4AAqkfJpSErQyxsjGaNxQSVDAABbEDADuUADkEDZgoFBwDKogOTzsvAD01VAAAsAEALSKkHzArd4MxFDBGZDERTmMDDmEqQxWblDkDMDyCqRE3KLiiCYAjABMtADaAERY2AcAugEAdKPcQA" rel="noreferrer">Playground Link</a></p>
<p>These comments are checked by the compiler, so that a compilation error occurs if the line following the comment would <em>not</em> otherwise raise a compilation error. It's recommended to add a description to explain why an error is expected.</p>
<p>TypeScript's approach has some limitations:</p>
<ul>
<li><code>// @ts-expect-error</code> comments are checked by the compiler, not the test runner, so tests are effectively run at build-time rather than alongside your other tests. This might also mean not taking advantage of the nice UI in your IDE for test results.</li>
<li>It's not possible to make more granular assertions about what kind of error should occur. For example, you might want to assert that a call like <code>foo(1, 2)</code> fails because the arguments are numbers, but the test would also pass if the call fails from having the wrong number of arguments. This is a solvable problem, but it would require inventing a syntax for describing errors.</li>
</ul>

CC BY-SA 4.0
2024-02-05T17:59:30.357Z
2024-02-05T18:20:22.707Z
2024-02-05T18:20:22.707Z
kaya3
CC BY-SA 4.0
2024-02-05T20:03:12.863Z
It would also require a stable error detection implementation in the compiler which is a challenge in practice if the language is still evolving.
feldentm
CC BY-SA 4.0
2024-02-05T22:02:00.070Z
@feldentm That's true, but it's fundamental to the problem of testing for compiler errors, and can itself be a reason for wanting such tests. If the compiler changes such that something which used to be an error is no longer an error, but it is *desired* to be an error, then the test is useful for indicating that some of the types need to change in order to make it an error again.
kaya3
CC BY-SA 4.0
2024-02-06T17:38:00.800Z
True, but if this exposed as a language feature, than changes have a much larger blast radius.
feldentm
<p><strong>Write a test harness that invokes the compiler</strong> with the failing code (in a separate file), and inspects the output to make sure it fails. See <a href="https://github.com/dtolnay/trybuild#compile-fail-tests" rel="noreferrer"><code>trybuild</code></a> for an example.</p>
<ul>
<li>Upside: works in any language</li>
<li>Downside: getting your project's dependencies may be hard; worst case, you may need to create a library sub-project. Invoking the compiler may be slow, especially because you probably need to invoke it multiple times (at best, once for every phase you have errors in; if the compiler is dumb and stops on a single error, you have to invoke it once for every test).</li>
</ul>
<hr />
<p>If the metaprogramming consists of running code at compile-time, you may be able to test it like any other code <strong>with a test-suite which also runs at compile-time</strong>.</p>
<p>E.g. in <strong>Racket</strong>, a macro is simply a function which takes and returns a syntax object. If you extract this function out of <a href="https://docs.racket-lang.org/reference/define.html#%28form._%28%28lib._racket%2Fprivate%2Fbase..rkt%29._define-syntax%29%29" rel="noreferrer"><code>define-syntax</code></a> into <a href="https://docs.racket-lang.org/reference/define.html#%28form._%28%28lib._racket%2Fprivate%2Fbase..rkt%29._define-for-syntax%29%29" rel="noreferrer"><code>define-for-syntax</code></a>, in a <a href="https://docs.racket-lang.org/reference/begin.html#%28form._%28%28quote._%7E23%7E25kernel%29._begin-for-syntax%29%29" rel="noreferrer"><code>begin-for-syntax</code></a> block you can call it directly. Here's an example of how to test a macro (derived from the <code>define-syntax</code> documentation):</p>
<pre class="lang-scheme prettyprint-override"><code>#lang racket/base
(define-for-syntax foo-impl
  (syntax-rules ()
    ((_ (a) ...) (printf &quot;~a\n&quot; (list a ...)))))
(define-syntax foo foo-impl)

; Usage
(foo (1) (2) (3) (4)) ; produces (printf &quot;~a\n&quot; (list 1 2 3 4))
; (foo 1 2 3 4) fails to compile

; How to test
(require (for-syntax racket/base rackunit))
(begin-for-syntax
  ; Test first usage
  (check-equal?
    (syntax-&gt;datum (foo-impl #'(foo (1) (2) (3) (4))))
    (syntax-&gt;datum #'(printf &quot;~a\n&quot; (list 1 2 3 4))))
  ; Test second usage (THIS IS WHERE WE TEST DOES NOT COMPILE)
  (check-exn exn:fail? (lambda () (foo-impl #'(foo 1 2 3 4)))))
</code></pre>
<p>Similarly in <strong>Zig</strong>, you can test any comptime function with an <a href="https://ziglang.org/documentation/master/std/#A;std:testing.expectError" rel="noreferrer"><code>expectError</code></a> which also runs at comptime (or at least I believe, I'm not as familiar with Zig and maybe there are some limitations which prevent comptime <code>expectError</code>...)</p>
<hr />
<p>AFAIK this feature is natively in <strong>Rust</strong> (documentation tests with the <a href="https://doc.rust-lang.org/rustdoc/write-documentation/documentation-tests.html#attributes" rel="noreferrer"><code>compile_fail</code> attribute</a>) and via libraries in <strong>Scala</strong> (<a href="https://www.scala-exercises.org/shapeless/type_checking" rel="noreferrer">Shapeless <code>illTyped</code></a>) and <strong>Haskell</strong> (<a href="https://github.com/CRogers/should-not-typecheck#should-not-typecheck" rel="noreferrer"><code>should-not-typecheck</code></a>).</p>

CC BY-SA 4.0
2024-02-05T16:03:00.197Z
2024-02-05T18:01:37.267Z
2024-02-05T18:01:37.267Z
tarzh
CC BY-SA 4.0
2024-02-05T23:38:15.253Z
See also https://docs.racket-lang.org/syntax/macro-testing.html#%28form._%28%28lib._syntax%2Fmacro-testing..rkt%29._convert-compile-time-error%29%29
D. Ben Knoble
# "Testing" that something does not compile in metaprogramming
Posted by: username_0 on 2024-02-05
Use case
Some languages offer techniques to ensure certain requirements at compile time. For example, rust has the NonZeroU32 type that will ensure at compile time you can't assign a zero value to it without checking first.
In more complex cases, it can be hard to see if some set of constraints actually restricts the output type as expected. Consider this type in typescript:
type Obj = {... some fields};

const getObj = async (id: number, fields: Array): Promise> =>
            GET(`/api/${ 
                id
            }?fields=${fields.join(',')}`);

// This should fail
getObj(12, ["bla"]).foo

This is intended to return a type with only the fields passed as strings returned (the rest won't even be fetched from the API). However, the way this works is a bit convoluted and it would be easy to make a mistake that would lead to it allowing accessing other fields, which would crash at runtime. I would want to ensure that can't happen. I can't add test cases for this though because if it failed it would prevent the entire test suit from compiling.
This is just an example, I'm looking for language design features that would allow doing this naturally, not advice on this specific case.
Some other examples of failures I might want to test for:
let x: NonZeroU32 = 0;

or
println!("{}"); // bad macro parameters

Question
Do any languages have systems that allow testing code that does not compile? How might such a system be designed?
I'm not looking for advice for this case specifically but more generally if other languages have methods to deal with similar situation.
**Tags:** static-typing testing meta-programming
> **Comments:**
> **username_1**: Maybe something like duck typing? I feel like it's a similar concept, where while it doesn't match a valid type/matches a disallowed type, the thing still quacks.  
> **username_0**: @Someone Wouldn't that defeat the whole purpose? The whole reason for defining the types like this is to make sure that it fails at compile time if it's not the right type  
> **username_1**: Maybe a strongly-typed language could have a `-B` "broken" option that uses duck typing? I mean, it seems close enough.  
> **username_0**: @Someone Feel free to add that as an answer. I don't know exactly what you mean but maybe it's what I'm looking for if expanded a bit more  
> **username_2**: @Someone if the language has name-based typing resorting to duck typing will influence type equality which will likely break other things too  
> **username_3**: The Scala community has longe since raised the more general question: given that our modern type systems are of comparable power to our modern programming languages, what should the tooling look like? What is the type-level equivalent of an IDE, a semantic highlighter, a debugger, a profiler, a test framework, a coverage analyzer? Note that I carefully phrased this as "the Scala community has raised the question", as I believe they have no found a good *answer* yet. But at least, they recognize the problem.  
---
## 6 answers
### Answer by username_7 on 2024-02-05. Score: 16
Write a test harness that invokes the compiler with the failing code (in a separate file), and inspects the output to make sure it fails. See trybuild for an example.

Upside: works in any language
Downside: getting your project's dependencies may be hard; worst case, you may need to create a library sub-project. Invoking the compiler may be slow, especially because you probably need to invoke it multiple times (at best, once for every phase you have errors in; if the compiler is dumb and stops on a single error, you have to invoke it once for every test).


If the metaprogramming consists of running code at compile-time, you may be able to test it like any other code with a test-suite which also runs at compile-time.
E.g. in Racket, a macro is simply a function which takes and returns a syntax object. If you extract this function out of define-syntax into define-for-syntax, in a begin-for-syntax block you can call it directly. Here's an example of how to test a macro (derived from the define-syntax documentation):
#lang racket/base
(define-for-syntax foo-impl
  (syntax-rules ()
    ((_ (a) ...) (printf "~a\n" (list a ...)))))
(define-syntax foo foo-impl)

; Usage
(foo (1) (2) (3) (4)) ; produces (printf "~a\n" (list 1 2 3 4))
; (foo 1 2 3 4) fails to compile

; How to test
(require (for-syntax racket/base rackunit))
(begin-for-syntax
  ; Test first usage
  (check-equal?
    (syntax->datum (foo-impl #'(foo (1) (2) (3) (4))))
    (syntax->datum #'(printf "~a\n" (list 1 2 3 4))))
  ; Test second usage (THIS IS WHERE WE TEST DOES NOT COMPILE)
  (check-exn exn:fail? (lambda () (foo-impl #'(foo 1 2 3 4)))))

Similarly in Zig, you can test any comptime function with an expectError which also runs at comptime (or at least I believe, I'm not as familiar with Zig and maybe there are some limitations which prevent comptime expectError...)

AFAIK this feature is natively in Rust (documentation tests with the compile_fail attribute) and via libraries in Scala (Shapeless illTyped) and Haskell (should-not-typecheck).
> **Comments:**
> **username_8** (2024-02-05): See also https://docs.racket-lang.org/syntax/macro-testing.html#%28form._%28%28lib._syntax%2Fmacro-testing..rkt%29._convert-compile-time-error%29%29  
---
### Answer by username_6 on 2024-02-05. Score: 10
TypeScript 3.9+ has a feature for this: // @ts-expect-error comments. In an example like yours, it would be:
type Obj = {bla: string, foo: string};

const getObj =  (id: number, fields: Array): Pick => {
    throw 'empty body';
}

// @ts-expect-error Property 'foo' should not exist
getObj(12, ["bla"]).foo

Playground Link
These comments are checked by the compiler, so that a compilation error occurs if the line following the comment would not otherwise raise a compilation error. It's recommended to add a description to explain why an error is expected.
TypeScript's approach has some limitations:

// @ts-expect-error comments are checked by the compiler, not the test runner, so tests are effectively run at build-time rather than alongside your other tests. This might also mean not taking advantage of the nice UI in your IDE for test results.
It's not possible to make more granular assertions about what kind of error should occur. For example, you might want to assert that a call like foo(1, 2) fails because the arguments are numbers, but the test would also pass if the call fails from having the wrong number of arguments. This is a solvable problem, but it would require inventing a syntax for describing errors.
> **Comments:**
> **username_2** (2024-02-05): It would also require a stable error detection implementation in the compiler which is a challenge in practice if the language is still evolving.  
> **username_6** (2024-02-05): @feldentm That's true, but it's fundamental to the problem of testing for compiler errors, and can itself be a reason for wanting such tests. If the compiler changes such that something which used to be an error is no longer an error, but it is *desired* to be an error, then the test is useful for indicating that some of the types need to change in order to make it an error again.  
> **username_2** (2024-02-06): True, but if this exposed as a language feature, than changes have a much larger blast radius.  
---
### Answer by username_4 on 2024-02-06. Score: 4
Do any languages have systems that allow testing code that does not compile?

Yes, several languages make it possible to compile code during runtime.

Perl has the eval method, which takes a string and compiles it during runtime (implicitly, before running said code). It makes it easily possible for the caller to work with any kind of error condition (compilation error or execution error).
Ruby has the Kernel.eval method which allows for much the same. As everything in Ruby it relies on the usual exception handling, leaving it up to the caller to see if there were compilation errors.
Java can do this too, albeit more involved and less integrated into the base language, by exposing the Java compiler as user-callable code.
For your own example, JavaScript has eval() too. (And for typescript there is a transpile() method.)
Even low level languages like C, which most definitely do not include a compiler in their executables, have been using the method of like simply calling the C compiler as a sub-process, on some piece of source code which has been created by the user program dynamically. This is heavily used, in, for example, the automake and similar build systems, to be able to test the compilation environment of some Unix tool and figure out which of a myriad of options said environment offers. This method can arguably used in every language which allows to run sub-processes, and where the compilation environment is available while the tests are running (which might or might not be trivial, if you are thinking about running your tests in an otherwise "lean" CI/CD runner environment, but certainly possible to achieve).

Using these tools, it is straightforward to write tests that make sure that the compilation fails, via some pseudo code:
test() {
  got_error = false;
  try {
    eval("this does not compile!");
  } catch (CompilationException) {    
    got_error = true;
  }
  assert(got_error);
}

Adjust by taste.
> **Comments:**
> **username_0** (2024-02-06): Nitpick: You can't `eval` typescript code. You'd need to run the typescript as a sub-process like your C answer  
> **username_4** (2024-02-06): Nitpick on your nitpick: there seems to be a `transpile()` which seems to be doing exactly that. I have added/clarified it. It's not the most important detail of the answer as OP was adamant in it being a generic question, with TS just an example; anyways. ;)  
---
### Answer by username_5 on 2024-02-05. Score: 3
In C++, and possibly other languages, we have the [Detection Idiom](https://benjaminbrock.net/blog/detection_idiom.php). This relies on [SFINAE](https://en.cppreference.com/w/cpp/language/sfinae) or Concepts to determine whether template code can be instantiated, yielding a `bool` which you can `static_assert()` as a test.

I don't have any examples to show, though I do intend to experiment a bit with this and perhaps create some utility code to make it easier to use.
> **Comments:**
> **username_5** (2024-02-10): Yes, that's the Concepts version of the detection idiom that I'm describing.  
---
### Answer by username_2 on 2024-02-05. Score: 1
Tyr had noCompile as a test category even in the first published version ([example](https://github.com/tyr-lang/test.conformance/blob/a7e5a0f44e4cedb5de2cbe54352862dd76e2ed6a/0.4.0/accept/ref/mar.tyr#L10)). The question can be split into two aspects.

The first is regular types that exclude certain values. Positive, nonzero and nonnull would be examples for such excluding subtypes. Implementation wise, they behave mostly like their base types except that they need to carry a proof that excluded values cannot reach values of the type. If this is not possible, the implementation can still resort to a runtime check or simply fail. While this category is mentioned explicitly in the question, it is not really about failing compilation. Personally, I would always have such checks as runtime checks because type systems usually lack the expressiveness, i.e. analysis quality, to make such features useful in real world code bases.

The second aspect is actual failing compilation.
Here, one must be aware what not compiling means for the compiler.
When it comes to syntax errors, there might be no reasonable way to recover from that situation and return to the boundaries of the noCompile expression or block or whatever is offered.
When it comes to type errors, the compiler needs thorough tracking of what made it reach that type.
Also, some errors are there to protect the compiler from running into an infinite loop. Getting out of such loop can mean to abort translation in a way that might not be suitable for recovery.
Finally, what might be unexpected, is that evaluation of malformed code can result in the compilers data structures such as the type context getting poisoned with nonsense. Depending on the error class, it can be hard to impossible to allow a compiler to execute the next phase or revert all its changes inside the noCompile context to allow it to produce a usable binary in the end. An example for such an issue is a malformed template type that results in an infinite materialization chain that gets aborted by a limit. This will itself cause an error in a phase requiring all partly materialized types to get fully materialized.

If you think about adding such a feature to your language, have a look at the implementation. If your error category is implemented by just adding an error message to the list and continuing with compilation afterwards, you might likely be fine. If you throw an exception and hope for some surrounding context to abort the member or wherever you can continue, you might exclude the category from such a feature.

TBH, I do not see why one would use a feature like noCompile outside of tests. The primary reason to allow regular programmers to write such tests is to ensure that APIs do not have certain properties like revealing internal state.
> **Comments:**
> **username_2** (2024-02-05): an example for inaccessible member test can be found here: https://github.com/tyr-lang/test.conformance/blob/a7e5a0f44e4cedb5de2cbe54352862dd76e2ed6a/0.5.0/accept/vcalls/mar.tyr#L18  
---
### Answer by username_1 on 2024-02-05. Score: 0
Compile-time errors due to typing are usually in strongly-typed languages. But if a type is too complicated for a compiler, perhaps a runtime type evaluation is needed.

I don't believe this exists in a language as of now (then again, there are *thousands* of esolangs), but perhaps some kind of compile/runtime flag?

# proposal (probably terrible)

Essentially, when compiling, there can be a `-B` (for **b**roken) flag that ignores type errors and switches to a duck typing stance. Essentially, while the types might *technically* not work, the untyped (or weakly typed) code still might, which is the whole idea behind duck typing.
> **Comments:**
> **username_0** (2024-02-05): How will this let me assert that some construct fails to compile?  
> **username_1** (2024-02-05): I…think I answered a slightly different question. Testing whether something compiles or not is something different entirely.  
---
