<p><a href="https://jqlang.github.io/jq/" rel="noreferrer">jq</a> is a command-line tool for filtering and transforming JSON data, in the vein of sed and awk, including a bespoke programming language for writing the filters. Its original author <a href="https://web.archive.org/web/20201111211031/http://stedolan.net/research/" rel="noreferrer">described it</a> as</p>
<blockquote>
<p>designed to be concise and easy to use for simple tasks, but if you dig a little deeper you'll find a featureful functional programming language hiding underneath</p>
</blockquote>
<p>The paradigm of that programming language is unusual: it <em>is</em> pure-functional, but the core model is of a single stream of filters selecting or producing values, connected by pipes (as in a shell pipeline). I won't rehash the manual, but a few core characteristics in brief:</p>
<ul>
<li>Each filter takes in one data value at a time and can produce any number of outputs, all of which are projected through the next filter individually.</li>
<li>Certain constructs capture many produced outputs into a single one, like the <a href="https://jqlang.github.io/jq/manual/#Arrayconstruction:%5B%5D" rel="noreferrer">array construction</a> circumfix <code>[ .articles[].title ]</code>, which produces one array containing all values from the filter inside.</li>
<li>Other constructions implicitly project single values into multiple entries, like using a multi-output filter within an <a href="https://jqlang.github.io/jq/manual/#ObjectConstruction:%7B%7D" rel="noreferrer">object constructor</a> <code>[1,2,3]|{x: .[]}</code>, which will produce multiple objects into the stream.</li>
<li>Inputs can be distributed across multiple filters with a <a href="https://jqlang.github.io/jq/manual/#Comma:," rel="noreferrer">comma</a> <code>.[] | tostring, sqrt</code>, producing the combined output of all of them interleaved.</li>
<li><a href="https://jqlang.github.io/jq/manual/#Variable/SymbolicBindingOperator:...as$identifier%7C..." rel="noreferrer">Variable bindings</a> can remember a specific value from an ancestor filter in the derivation of the current value — one of the ways it's visibly not just a flat stream. </li>
</ul>
<p>Some properties silently fall out of these rules unnoticed, like <code>[1,2,3]</code> constructing an array of three numbers. The <a href="https://jqlang.github.io/jq/manual/" rel="noreferrer">manual</a> provides a more complete description of the language semantics, and there could well be corner cases or other features that are relevant.</p>
<p><strong>Is there a name for this evaluation model and paradigm</strong>, that I could use to find more instances of it?</p>
<p>There are strong similarities to the list monad, with <code>|</code> as the bind operation, and it is clearly a data-flow system in many respects, but I'd like to find the <strong>term or terms that encapsulate</strong> or collectively identify the paradigm of the language. It may well (likely will) be a conjunction of terms, though if there's a single established name that'd be great too.</p>

<terminology><programming-paradigms><jq>
Does the evaluation model & programming paradigm used in jq have a name?
CC BY-SA 4.0
2023-06-07T11:59:20.237Z
Is this on-topic? I’m not confident enough that it is to upvote it, but I’m not confident enough that it isn’t to VTC.
Bbrk24
CC BY-SA 4.0
2023-06-07T13:28:46.663Z
The relevant terms seem to be [dataflow programming](https://en.wikipedia.org/wiki/Dataflow_programming), [pipelines](https://en.wikipedia.org/wiki/Pipeline_(software)) and [stream processing](https://en.wikipedia.org/wiki/Stream_processing), but these terms mostly appear in your question. So I'm not sure this is worthy of an answer.
kaya3
CC BY-SA 4.0
2023-06-07T17:11:53.387Z
I'm interested in this but the question seems a bit shallow for the amount of research on the subject it already has. I recently had to reimplement a jq-like language for queries and having resources on this paradigm would be great
kouta-kun
CC BY-SA 4.0
2023-06-07T19:44:31.927Z
I personally think this is on-topic. It is about a programming paradigm. Also, I'm not sure of the name myself and want to know more about it. So maybe the question author can add a question regarding resources about the paradigm?
Gavin D. Howard
CC BY-SA 4.0
2023-06-07T22:43:11.430Z
@kaya3 It's possible that "dataflow pipeline" is the best available term, but the answer I'm hoping for is something like "this is *x*-based *y* dataflow programming", with X and Y accounting for e.g. the preservation of derivations whenever it branches rather than being a flat pipeline. That is, that there's a specific family of pipeline processing that this is, not just within the broad category - it's not semantically parallel to Unix pipes, for example. I'm not going to be shocked if there isn't one, but I figure it's more likely than not that there's *something* to identify them.
Michael Homer
CC BY-SA 4.0
2023-06-07T22:47:40.610Z
@Bbrk24 Perhaps questions seeking terminology will be determined as off-topic generally (there is [a meta post about this](https://languagedesign.meta.stackexchange.com/q/151/133) suggesting they are ok currently), but otherwise I think it's in scope. We'll see how things go, though, maybe ultimately we decide they don't fit. "What is this thing in a PL?" seems like an objectively-answerable question within the range of the site at least.
Michael Homer
CC BY-SA 4.0
2023-06-08T00:01:00.710Z
Yes, it's not like Unix pipes ─ the more specific term from that the Wikipedia page is [object pipelines](https://en.wikipedia.org/wiki/Pipeline_(software)#Object_pipelines), i.e. the data passed through the pipeline is some kind of object or data structure, rather than raw text. Is that enough for an answer? It still feels like the question is way more informative than the answer.
kaya3
CC BY-SA 4.0
2023-06-08T02:05:59.370Z
@kaya3 An object pipeline (like PowerShell) is not, I think, the same model as jq — those are generally flat pipelines as far as I know, where each output object just goes into the one stream that the next filter takes in, but jq remembers how you got there — it's more list monad than pipeline. Consider something like `jq -n '1,2,3 | {x:., y: (. * 2)} | .x as $a | .x, .y | "\($a) \(.)"'`, where `$a` holds the corresponding value from earlier. You can't write that in PowerShell, because each filter only runs one time for the whole pipeline, just like a Unix pipeline.
Michael Homer
CC BY-SA 4.0
2023-06-08T12:14:27.790Z
Filters may return multiple values. That reminds me of functional logic languages like [Curry](https://curry.pages.ps.informatik.uni-kiel.de/curry-lang.org/).
alephalpha
CC BY-SA 4.0
2023-06-13T09:09:28.407Z
XPath is probably relevant, since it has a similar data model for querying XML. And it’s much more thoroughly specified, so somewhere in that hefty amount of text might be a mention of terminology and prior art.
Jon Purdy
CC BY-SA 4.0
2023-07-02T11:25:48.367Z
It feels related to multi-valued generators like Icon language.  It adds Unix-inspired pipelining syntax where "stdin" is tacitly provided to a whole expression, but I'm not sure how much that's syntactical vs. semantic?  I thought implicit iteration only happens over stdin, but comma operator proves me wrong: `echo 0 | jq '(3, 4) + (10, 20)'` yields 13 14 23 24.
Beni Cherniavsky-Paskin
<blockquote>
<p>There are strong similarities to the list monad, with <code>|</code> as the bind operation, and it is clearly a data-flow system in many respects, but I'd like to find the <strong>term or terms that encapsulate</strong> or collectively identify the paradigm of the language.</p>
</blockquote>
<p>In the academic literature, this ability for a single expression to evaluate to multiple values is known as <em>nondeterministic choice</em>, often shortened to just <em>nondeterminism</em>. (Yes, this can be confusing, as it has little to do with what most working programmers would call “nondeterminism”.) It’s sometimes also called <em>ambiguous choice</em>, though this is an older term that has mostly fallen out of usage.</p>
<p>To my knowledge, this concept was first introduced by John McCarthy (creator of Lisp) in <a href="https://www.sciencedirect.com/science/article/abs/pii/S0049237X09700990" rel="noreferrer">A Basis for a Mathematical Theory of Computation</a> (<a href="https://dspace.mit.edu/bitstream/handle/1721.1/6099/AIM-031.pdf" rel="noreferrer">pdf</a>) from 1961. McCarthy defines a function <code>amb(x, y)</code> that has “possible values” <code>x</code> and <code>y</code>. When used within a larger expression, the expression can be evaluated by choosing either value, much like ±x can be interpreted as either +x or −x, even when incorporated into a larger expression.</p>
<p>The term “nondeterminism” is used to describe these sorts of expressions because, in a sense, you can think of <code>amb(1, 2)</code> as “nondeterministically” evaluating to <code>1</code> or to <code>2</code>. In many systems, like <code>jq</code> or the Haskell list monad, the program’s result involves collecting <em>all possible</em> results of a given expression, but from within the expression itself, these “parallel universes” cannot be directly observed. The timeline is simply split.</p>
<p>More modern treatments provide a variety of different interpretations and formulations of nondeterministic choice, so I will not attempt to summarize them here; you can <a href="https://scholar.google.com/scholar?q=mccarthy%20nondeterminism" rel="noreferrer">search Google Scholar for “McCarthy nondeterminism”</a> and get plenty of results. However, one particularly topical example of nondeterminism in the literature is the Verse language currently being developed by Epic Games, which (perhaps surprisingly) <a href="https://simon.peytonjones.org/assets/pdfs/verse-conf.pdf" rel="noreferrer">is at its core a functional logic language with nondeterministic choice very deeply baked-in</a>.</p>
<p>Of course, <code>jq</code> includes much more than just nondeterministic choice. It also includes stream processing, a combinator language for building up parts of the program and delimiting the parts that get split into “parallel universes”, a broadcasting-like mechanism for implicitly introducing choice points, and feedback mechanisms that resemble those in <a href="https://en.wikipedia.org/wiki/Functional_reactive_programming" rel="noreferrer">functional reactive programming</a>. But I think the core of your question—the semi-magical way that single expressions seem to produce many values—is most closely identified with nondeterminism.</p>

CC BY-SA 4.0
2023-06-28T04:45:42.317Z
2023-06-28T04:45:42.317Z
Alexis King
# Does the evaluation model & programming paradigm used in jq have a name?
Posted by: username_0 on 2023-06-07
[jq](https://jqlang.github.io/jq/) is a command-line tool for filtering and transforming JSON data, in the vein of sed and awk, including a bespoke programming language for writing the filters. Its original author [described it](https://web.archive.org/web/20201111211031/http://stedolan.net/research/) as

> designed to be concise and easy to use for simple tasks, but if you dig a little deeper you'll find a featureful functional programming language hiding underneath

The paradigm of that programming language is unusual: it *is* pure-functional, but the core model is of a single stream of filters selecting or producing values, connected by pipes (as in a shell pipeline). I won't rehash the manual, but a few core characteristics in brief:

* Each filter takes in one data value at a time and can produce any number of outputs, all of which are projected through the next filter individually.
* Certain constructs capture many produced outputs into a single one, like the [array construction](https://jqlang.github.io/jq/manual/#Arrayconstruction:%5B%5D) circumfix `[ .articles[].title ]`, which produces one array containing all values from the filter inside.
* Other constructions implicitly project single values into multiple entries, like using a multi-output filter within an [object constructor](https://jqlang.github.io/jq/manual/#ObjectConstruction:%7B%7D) `[1,2,3]|{x: .[]}`, which will produce multiple objects into the stream.
* Inputs can be distributed across multiple filters with a [comma](https://jqlang.github.io/jq/manual/#Comma:,) `.[] | tostring, sqrt`, producing the combined output of all of them interleaved.
* [Variable bindings](https://jqlang.github.io/jq/manual/#Variable/SymbolicBindingOperator:...as$identifier%7C...) can remember a specific value from an ancestor filter in the derivation of the current value — one of the ways it's visibly not just a flat stream.

Some properties silently fall out of these rules unnoticed, like `[1,2,3]` constructing an array of three numbers. The [manual](https://jqlang.github.io/jq/manual/) provides a more complete description of the language semantics, and there could well be corner cases or other features that are relevant.

**Is there a name for this evaluation model and paradigm**, that I could use to find more instances of it?

There are strong similarities to the list monad, with `|` as the bind operation, and it is clearly a data-flow system in many respects, but I'd like to find the **term or terms that encapsulate** or collectively identify the paradigm of the language. It may well (likely will) be a conjunction of terms, though if there's a single established name that'd be great too.
**Tags:** terminology programming-paradigms jq
> **Comments:**
> **username_1**: Is this on-topic? I’m not confident enough that it is to upvote it, but I’m not confident enough that it isn’t to VTC.  
> **username_2**: The relevant terms seem to be [dataflow programming](https://en.wikipedia.org/wiki/Dataflow_programming), [pipelines](https://en.wikipedia.org/wiki/Pipeline_(software)) and [stream processing](https://en.wikipedia.org/wiki/Stream_processing), but these terms mostly appear in your question. So I'm not sure this is worthy of an answer.  
> **username_3**: I'm interested in this but the question seems a bit shallow for the amount of research on the subject it already has. I recently had to reimplement a jq-like language for queries and having resources on this paradigm would be great  
> **username_4**: I personally think this is on-topic. It is about a programming paradigm. Also, I'm not sure of the name myself and want to know more about it. So maybe the question author can add a question regarding resources about the paradigm?  
> **username_0**: @kaya3 It's possible that "dataflow pipeline" is the best available term, but the answer I'm hoping for is something like "this is *x*-based *y* dataflow programming", with X and Y accounting for e.g. the preservation of derivations whenever it branches rather than being a flat pipeline. That is, that there's a specific family of pipeline processing that this is, not just within the broad category - it's not semantically parallel to Unix pipes, for example. I'm not going to be shocked if there isn't one, but I figure it's more likely than not that there's *something* to identify them.  
> **username_0**: @Bbrk24 Perhaps questions seeking terminology will be determined as off-topic generally (there is [a meta post about this](https://languagedesign.meta.stackexchange.com/q/151/133) suggesting they are ok currently), but otherwise I think it's in scope. We'll see how things go, though, maybe ultimately we decide they don't fit. "What is this thing in a PL?" seems like an objectively-answerable question within the range of the site at least.  
> **username_2**: Yes, it's not like Unix pipes ─ the more specific term from that the Wikipedia page is [object pipelines](https://en.wikipedia.org/wiki/Pipeline_(software)#Object_pipelines), i.e. the data passed through the pipeline is some kind of object or data structure, rather than raw text. Is that enough for an answer? It still feels like the question is way more informative than the answer.  
> **username_0**: @kaya3 An object pipeline (like PowerShell) is not, I think, the same model as jq — those are generally flat pipelines as far as I know, where each output object just goes into the one stream that the next filter takes in, but jq remembers how you got there — it's more list monad than pipeline. Consider something like `jq -n '1,2,3 | {x:., y: (. * 2)} | .x as $a | .x, .y | "\($a) \(.)"'`, where `$a` holds the corresponding value from earlier. You can't write that in PowerShell, because each filter only runs one time for the whole pipeline, just like a Unix pipeline.  
> **username_5**: Filters may return multiple values. That reminds me of functional logic languages like [Curry](https://curry.pages.ps.informatik.uni-kiel.de/curry-lang.org/).  
> **username_6**: XPath is probably relevant, since it has a similar data model for querying XML. And it’s much more thoroughly specified, so somewhere in that hefty amount of text might be a mention of terminology and prior art.  
> **username_7**: It feels related to multi-valued generators like Icon language.  It adds Unix-inspired pipelining syntax where "stdin" is tacitly provided to a whole expression, but I'm not sure how much that's syntactical vs. semantic?  I thought implicit iteration only happens over stdin, but comma operator proves me wrong: `echo 0 | jq '(3, 4) + (10, 20)'` yields 13 14 23 24.  
---
## 1 answers
### Answer by username_8 on 2023-06-28. Score: 9
> There are strong similarities to the list monad, with `|` as the bind operation, and it is clearly a data-flow system in many respects, but I'd like to find the **term or terms that encapsulate** or collectively identify the paradigm of the language.

In the academic literature, this ability for a single expression to evaluate to multiple values is known as *nondeterministic choice*, often shortened to just *nondeterminism*. (Yes, this can be confusing, as it has little to do with what most working programmers would call “nondeterminism”.) It’s sometimes also called *ambiguous choice*, though this is an older term that has mostly fallen out of usage.

To my knowledge, this concept was first introduced by John McCarthy (creator of Lisp) in [A Basis for a Mathematical Theory of Computation](https://www.sciencedirect.com/science/article/abs/pii/S0049237X09700990) ([pdf](https://dspace.mit.edu/bitstream/handle/1721.1/6099/AIM-031.pdf)) from 1961. McCarthy defines a function `amb(x, y)` that has “possible values” `x` and `y`. When used within a larger expression, the expression can be evaluated by choosing either value, much like ±x can be interpreted as either +x or −x, even when incorporated into a larger expression.

The term “nondeterminism” is used to describe these sorts of expressions because, in a sense, you can think of `amb(1, 2)` as “nondeterministically” evaluating to `1` or to `2`. In many systems, like `jq` or the Haskell list monad, the program’s result involves collecting *all possible* results of a given expression, but from within the expression itself, these “parallel universes” cannot be directly observed. The timeline is simply split.

More modern treatments provide a variety of different interpretations and formulations of nondeterministic choice, so I will not attempt to summarize them here; you can [search Google Scholar for “McCarthy nondeterminism”](https://scholar.google.com/scholar?q=mccarthy%20nondeterminism) and get plenty of results. However, one particularly topical example of nondeterminism in the literature is the Verse language currently being developed by Epic Games, which (perhaps surprisingly) [is at its core a functional logic language with nondeterministic choice very deeply baked-in](https://simon.peytonjones.org/assets/pdfs/verse-conf.pdf).

Of course, `jq` includes much more than just nondeterministic choice. It also includes stream processing, a combinator language for building up parts of the program and delimiting the parts that get split into “parallel universes”, a broadcasting-like mechanism for implicitly introducing choice points, and feedback mechanisms that resemble those in [functional reactive programming](https://en.wikipedia.org/wiki/Functional_reactive_programming). But I think the core of your question—the semi-magical way that single expressions seem to produce many values—is most closely identified with nondeterminism.
---
