<p>In APL, you don't need to write loops because scalar operations are automatically &quot;broadcast&quot; (mapped and zipped) across arrays.</p>
<p>For example a grayscale image could be a 2D array, which is like a function (int, int -&gt; float). If I add a 1D array to this, it zips along the x axis, and broadcasts along the y axis. In effect, the 1D array represents an x-varying signal, which is automatically lifted to be constant with respect to y, and the zipped with the image. If I wanted to interpret the array as a y-varying signal, I'd need to transpose it or use the <a href="https://aplwiki.com/wiki/Rank_(operator)" rel="noreferrer">rank operator</a> to be more explicit about which axes are zipped.</p>
<p>I find it hard to keep track of the axes because they don't have names. Instead I'd like arrays to be more like functions with keyword arguments: the image would be (x: int, y: int -&gt; float), an x-varying signal would be (x: int -&gt; float), and a y-varying signal would be (y: int -&gt; float).</p>
<p>I think this is what dataframes (like <a href="https://pandas.pydata.org/" rel="noreferrer">Pandas</a>) are. Many unary operations are implicitly mapped over the array, and many binary operations are implicitly natural-joined on some special named columns called &quot;index&quot; columns. I like this paradigm, but there's often friction with the host language (Python) because not all operations are overloadable: for example Python's built-in string and regex functions are not implicitly mapped over Pandas dataframes.</p>
<p>Is there a language where &quot;everything is a dataframe&quot;? Or, one where dataframes are central, and are used instead of loops (like arrays in APL)?</p>
<ul>
<li><p>I think dataframes in R are provided by a library, so I assume there is similar friction there as with Pandas in Python.</p>
</li>
<li><p>It looks like <a href="https://futhark-lang.org/" rel="noreferrer">Futhark</a> is not an example, because it <a href="https://futhark-lang.org/blog/2021-12-19-past-and-present.html#broadcasting" rel="noreferrer">doesn't have rank polymorphism</a>.</p>
</li>
<li><p><a href="https://github.com/google-research/dex-lang" rel="noreferrer">Dex</a> solves this problem in a different way, by having you name the axes each time you iterate over them, in a comprehension. While I like this, I don't count it as an answer to this question, because I'm looking for something more point-free like APL.</p>
</li>
<li><p>Pyret has <a href="https://papl.cs.brown.edu/2020/intro-tabular-data.html" rel="noreferrer">tables built in to the language</a>, but it doesn't look like they are implicitly broadcast.</p>
</li>
</ul>

<apl><programming-paradigms>
Is there a language where "everything is a dataframe"?
CC BY-SA 4.0
2024-04-13T00:39:48.117Z
Languages where everything is a vector can still represent scalar values as vectors of size 1. But in a language where everything is a dataframe, scalar representation isn't so clear -- what would the scalar's "name" be?
tarzh
CC BY-SA 4.0
2024-04-13T00:51:16.293Z
I think relational databases technically fit this answer, because everything is a **table** and tables are equivalent to a data-frames. One could write a language with syntax like you described (`-table`, `table1 + table2`) which transpiles into SQL queries and views.
tarzh
CC BY-SA 4.0
2024-04-13T10:06:08.800Z
@tarzh In the original meaning, everything in a relational database is a *relation*, which is an unordered set of *tuples* (and [nothing to do with *relationships* in the way people often assume](https://softwareengineering.stackexchange.com/a/422044/96713)); SQL's "tables" and "columns" are a particular pragmatic implementation of the general idea, but *relational algebra* probably comes closer to the theme of this question.
IMSoP
CC BY-SA 4.0
2024-04-18T03:37:26.750Z
Your perception of R is incorrect: data frames are built in to the language (no imports required), and they use the languages built in vector type for their columns. In my experience, this provides a far smoother experience than the situation with pandas in Python. R doesn't use a point free style in the way APL does, but operations are automatically broadcast in one dimension, and the built in [apply functions](https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/apply) make broadcasting multi-dimenension operations quite natural. You rarely write loops in R.
setholopolus
CC BY-SA 4.0
2024-04-18T17:45:05.590Z
@tarzh I think a scalar would be a 1-element pandas.Series, where the index is an empty tuple. (Like a function of zero arguments.) Maybe a better framing would be "a language where everything is a Series", and a dataframe is a struct-valued series.
dpercy
<p>I think that the closest language to what you are asking for is actually <a href="https://en.wikipedia.org/wiki/Stata" rel="nofollow noreferrer">Stata</a>, a language that most computer scientists have never heard of, because it is mostly used by working economists (and some people in sociology, biomedicine, and epidemiology, according to Wikipedia).</p>
<p>The typical use of stata is that you load a dataset, and then all commands you run in the language are assumed to be over that data set, for example simply running</p>
<pre><code>summarize
</code></pre>
<p>gives a summary of the dataset, and</p>
<pre><code>correlate
</code></pre>
<p>gives the correlation matrix for all variables in the data set.</p>
<p>Of course there are also more complex commands that take arguments as well, but it is still assumed that every operation is implicitly over 'the dataset'.</p>

CC BY-SA 4.0
2024-04-17T19:36:50.030Z
2024-04-17T19:36:50.030Z
setholopolus
CC BY-SA 4.0
2024-04-17T23:28:12.050Z
Could you [edit] in some elaboration on how "everything is a dataframe" in Stata? That doesn't really come out anywhere currently.
Michael Homer
<p>There's an interesting language from the 1980s called <a href="https://en.wikipedia.org/wiki/Lucid_(programming_language)" rel="nofollow noreferrer">Lucid</a> where every value is actually a &quot;stream&quot; of values and operations operate on those streams elementwise, along with special operators to expand/compress/shift streams relative to each other.  Later versions of the language added the ability to add more dimensions in addition to the default &quot;time&quot; dimension and operators to &quot;rotate&quot; through dimensions.</p>

CC BY-SA 4.0
2024-04-14T00:51:03.177Z
2024-04-14T00:51:03.177Z
Chris Dodd
# Is there a language where "everything is a dataframe"?
Posted by: username_0 on 2024-04-12
In APL, you don't need to write loops because scalar operations are automatically "broadcast" (mapped and zipped) across arrays.

For example a grayscale image could be a 2D array, which is like a function (int, int -> float). If I add a 1D array to this, it zips along the x axis, and broadcasts along the y axis. In effect, the 1D array represents an x-varying signal, which is automatically lifted to be constant with respect to y, and the zipped with the image. If I wanted to interpret the array as a y-varying signal, I'd need to transpose it or use the [rank operator](https://aplwiki.com/wiki/Rank_(operator)) to be more explicit about which axes are zipped.

I find it hard to keep track of the axes because they don't have names. Instead I'd like arrays to be more like functions with keyword arguments: the image would be (x: int, y: int -> float), an x-varying signal would be (x: int -> float), and a y-varying signal would be (y: int -> float).

I think this is what dataframes (like [Pandas](https://pandas.pydata.org/)) are. Many unary operations are implicitly mapped over the array, and many binary operations are implicitly natural-joined on some special named columns called "index" columns. I like this paradigm, but there's often friction with the host language (Python) because not all operations are overloadable: for example Python's built-in string and regex functions are not implicitly mapped over Pandas dataframes.

Is there a language where "everything is a dataframe"? Or, one where dataframes are central, and are used instead of loops (like arrays in APL)?

* I think dataframes in R are provided by a library, so I assume there is similar friction there as with Pandas in Python.
* It looks like [Futhark](https://futhark-lang.org/) is not an example, because it [doesn't have rank polymorphism](https://futhark-lang.org/blog/2021-12-19-past-and-present.html#broadcasting).
* [Dex](https://github.com/google-research/dex-lang) solves this problem in a different way, by having you name the axes each time you iterate over them, in a comprehension. While I like this, I don't count it as an answer to this question, because I'm looking for something more point-free like APL.
* Pyret has [tables built in to the language](https://papl.cs.brown.edu/2020/intro-tabular-data.html), but it doesn't look like they are implicitly broadcast.
**Tags:** apl programming-paradigms
> **Comments:**
> **username_1**: Languages where everything is a vector can still represent scalar values as vectors of size 1. But in a language where everything is a dataframe, scalar representation isn't so clear -- what would the scalar's "name" be?  
> **username_1**: I think relational databases technically fit this answer, because everything is a **table** and tables are equivalent to a data-frames. One could write a language with syntax like you described (`-table`, `table1 + table2`) which transpiles into SQL queries and views.  
> **username_2**: @tarzh In the original meaning, everything in a relational database is a *relation*, which is an unordered set of *tuples* (and [nothing to do with *relationships* in the way people often assume](https://softwareengineering.stackexchange.com/a/422044/96713)); SQL's "tables" and "columns" are a particular pragmatic implementation of the general idea, but *relational algebra* probably comes closer to the theme of this question.  
> **username_3**: Your perception of R is incorrect: data frames are built in to the language (no imports required), and they use the languages built in vector type for their columns. In my experience, this provides a far smoother experience than the situation with pandas in Python. R doesn't use a point free style in the way APL does, but operations are automatically broadcast in one dimension, and the built in [apply functions](https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/apply) make broadcasting multi-dimenension operations quite natural. You rarely write loops in R.  
> **username_0**: @tarzh I think a scalar would be a 1-element pandas.Series, where the index is an empty tuple. (Like a function of zero arguments.) Maybe a better framing would be "a language where everything is a Series", and a dataframe is a struct-valued series.  
---
## 2 answers
### Answer by username_3 on 2024-04-17. Score: 3
I think that the closest language to what you are asking for is actually Stata, a language that most computer scientists have never heard of, because it is mostly used by working economists (and some people in sociology, biomedicine, and epidemiology, according to Wikipedia).
The typical use of stata is that you load a dataset, and then all commands you run in the language are assumed to be over that data set, for example simply running
summarize

gives a summary of the dataset, and
correlate

gives the correlation matrix for all variables in the data set.
Of course there are also more complex commands that take arguments as well, but it is still assumed that every operation is implicitly over 'the dataset'.
> **Comments:**
> **username_4** (2024-04-17): Could you [edit] in some elaboration on how "everything is a dataframe" in Stata? That doesn't really come out anywhere currently.  
---
### Answer by username_5 on 2024-04-14. Score: 3
There's an interesting language from the 1980s called [Lucid](https://en.wikipedia.org/wiki/Lucid_(programming_language)) where every value is actually a "stream" of values and operations operate on those streams elementwise, along with special operators to expand/compress/shift streams relative to each other. Later versions of the language added the ability to add more dimensions in addition to the default "time" dimension and operators to "rotate" through dimensions.
---
