<p>I've inherited maintenance duties for a domain-specific language.  This language started life forty years ago as a quick way to insert the results of mathematical operations into text documents, and has grown by accretion since then.</p>
<p>The first thing I'd like to do is document the syntax of the language -- the <em>real</em> syntax, not the &quot;handed down by word-of-mouth for forty years&quot; syntax.  Is there a method for doing this that's easier than trying to infer the syntax from 10,000 lines of poorly-commented parser-interpreter?</p>

<domain-specific-languages><documentation>
Documenting the syntax of an existing language
CC BY-SA 4.0
2023-05-23T20:48:18.557Z
To be clear, the problem here is that you have the code which implements the DSL, and you want to figure out what its syntax is from the code? Not you have the syntax and you need to write a formal specification for it.
kaya3
CC BY-SA 4.0
2023-05-23T21:04:50.343Z
is there any existing code written in the language you can look at? If there is, I think that's a good way to start. Look at existing code, look at specific features, and try to trace those to code in the parser-interpreter, and just observe what that code does when it is run. Try extracting code into smaller programs, getting those to be well formed, and then observing their behaviour. I'm holding off on writing an answer post because I'd like to see what people think of the quality of this question and if it's a good fit for this site.
starball
CC BY-SA 4.0
2023-05-23T21:39:05.400Z
I’m voting to close this question because it's not about programming language design and implementation. We aren't redesigning the language here, and we already have an implementation (even if poorly written.)
Isaiah
CC BY-SA 4.0
2023-05-23T21:40:39.587Z
A question about how to represent the syntax in a formal or useful way given knowledge of the syntax would be on-topic, but decoding the syntax from existing code is neither a design nor implementation question and falls more in the realm of software engineering.
Isaiah
CC BY-SA 4.0
2023-05-23T21:49:19.293Z
also possibly related, but I'm really not sure, https://reverseengineering.stackexchange.com/help/on-topic
starball
CC BY-SA 4.0
2023-05-23T22:33:50.150Z
@kaya3, formal specification will be the next step, hopefully followed by replacing the parser with one that doesn't crash if you look at it funny.
Mark
<p>I once had the same task but simpler: I needed to reverse-engineer TypeScript grammar in the form of an ANTLR grammar. You might think TypeScript <em>has</em> a grammar, at least informal one, but no, it's outdated for ages and new syntax constructs were only described in release notes. By the way I bet ANTLR grammar is still incomplete though I haven't checked. :)</p>
<p>Anyway, what made it simpler than your case is TypeScript has a huge set of tests. Thus I ran my incomplete grammar over tests and looked at the examples that failed to parse, then updated the grammar. And then another loop of the same algorithm.</p>
<p>I guess you don't have tests, but generating a set of tests given a grammar sounds like something a compiler fuzzer can do, though I can't provide any links. But given such set of valid examples according to your candidate grammar, you can run actual interpreter or compiler on them to see which are indeed correct and which are not. Then update the grammar accordingly. Rinse and repeat.</p>

CC BY-SA 4.0
2023-05-24T07:27:32.230Z
2023-05-24T07:27:32.230Z
Alex Chichigin
CC BY-SA 4.0
2023-05-24T07:30:30.500Z
if you're going to fuzz for valid tests to revers-engineer a grammar, why not just fuzz for the grammar directly? (that's what I was going to propose... more as a joke though. I'm not really more efficient than just trying to read and understand the source)
starball
CC BY-SA 4.0
2023-05-24T07:36:20.670Z
I suspect writing down grammar by hand guided by parser failures in (presumably) an _intelligent_ way is a more efficient fuzzing strategy... 
Alex Chichigin
# Documenting the syntax of an existing language
Posted by: username_0 on 2023-05-23
I've inherited maintenance duties for a domain-specific language. This language started life forty years ago as a quick way to insert the results of mathematical operations into text documents, and has grown by accretion since then.

The first thing I'd like to do is document the syntax of the language -- the *real* syntax, not the "handed down by word-of-mouth for forty years" syntax. Is there a method for doing this that's easier than trying to infer the syntax from 10,000 lines of poorly-commented parser-interpreter?
**Tags:** domain-specific-languages documentation
> **Comments:**
> **username_1**: To be clear, the problem here is that you have the code which implements the DSL, and you want to figure out what its syntax is from the code? Not you have the syntax and you need to write a formal specification for it.  
> **username_2**: is there any existing code written in the language you can look at? If there is, I think that's a good way to start. Look at existing code, look at specific features, and try to trace those to code in the parser-interpreter, and just observe what that code does when it is run. Try extracting code into smaller programs, getting those to be well formed, and then observing their behaviour. I'm holding off on writing an answer post because I'd like to see what people think of the quality of this question and if it's a good fit for this site.  
> **username_3**: I’m voting to close this question because it's not about programming language design and implementation. We aren't redesigning the language here, and we already have an implementation (even if poorly written.)  
> **username_3**: A question about how to represent the syntax in a formal or useful way given knowledge of the syntax would be on-topic, but decoding the syntax from existing code is neither a design nor implementation question and falls more in the realm of software engineering.  
> **username_2**: also possibly related, but I'm really not sure, https://reverseengineering.stackexchange.com/help/on-topic  
> **username_0**: @kaya3, formal specification will be the next step, hopefully followed by replacing the parser with one that doesn't crash if you look at it funny.  
---
## 1 answers
### Answer by username_4 on 2023-05-24. Score: 5
I once had the same task but simpler: I needed to reverse-engineer TypeScript grammar in the form of an ANTLR grammar. You might think TypeScript *has* a grammar, at least informal one, but no, it's outdated for ages and new syntax constructs were only described in release notes. By the way I bet ANTLR grammar is still incomplete though I haven't checked. :)

Anyway, what made it simpler than your case is TypeScript has a huge set of tests. Thus I ran my incomplete grammar over tests and looked at the examples that failed to parse, then updated the grammar. And then another loop of the same algorithm.

I guess you don't have tests, but generating a set of tests given a grammar sounds like something a compiler fuzzer can do, though I can't provide any links. But given such set of valid examples according to your candidate grammar, you can run actual interpreter or compiler on them to see which are indeed correct and which are not. Then update the grammar accordingly. Rinse and repeat.
> **Comments:**
> **username_2** (2023-05-24): if you're going to fuzz for valid tests to revers-engineer a grammar, why not just fuzz for the grammar directly? (that's what I was going to propose... more as a joke though. I'm not really more efficient than just trying to read and understand the source)  
> **username_4** (2023-05-24): I suspect writing down grammar by hand guided by parser failures in (presumably) an _intelligent_ way is a more efficient fuzzing strategy...   
---
