<p>I've been a user of SQL for a long time now, and it is regarded as extremely well-adapted for its various and widespread uses in data processing. I think you'll hardly find a business that owns a computer, where SQL is not somehow involved in their applications.</p>
<p>It has various characteristics, including (the ones I think are relevant for this question...):</p>
<ul>
<li>tables as first-class types with associated operators (Join, aggregate, etc.)</li>
<li>every scalar type has a first-class null value</li>
<li>a particular (and justifiable) system of how operators handle null values.</li>
</ul>
<p>There have been passing fads about &quot;object databases&quot;, about innovations which question the necessity of the functionality which SQL engines provide, and much commentary over the years about the &quot;impedance mismatch&quot; between SQL and OO languages.</p>
<p>And I can remember earnest attempts to marry SQL together with other programming languages and GUIs, such as Microsoft Access - which married a database engine called Jet, an SQL language based on ANSI-92, the VBA language (practically equivalent to VB6), and a GUI designer/IDE (which for many simple cases at least, could create and represent a database application with &quot;no code&quot;).</p>
<p>Access is still extant, but moribund now. Instead focus has shifted onto more modern languages like C#, and ORM layers like Entity Framework which attempt to concentrate the &quot;impedance mismatch&quot; into a translation layer and resolve it there. Belatedly, the C# language got things like nullable types, too.</p>
<p>For me, amongst the many problems with Access was that VBA had no native nullable types - VBA wasn't designed specifically for this application. What VBA had instead was the generic &quot;Variant&quot; type, which besides storing any type could also store an untyped null value. You could also have arrays of unnamed variants. But there were no built-in table types, no table operators, the built-in null handling was lacklustre and not fully commensurate with SQL. The marriage was workable for the time, but you could very much notice the seams in it.</p>
<p><strong>Question</strong></p>
<p>What I'm confused about, and which brings me to my question, is why no system of programming has ever been devised that simply attempts to run with SQL concepts (like listed above) throughout the design, right through to the front-end, thus resolving the notorious &quot;impedance mismatch&quot;?</p>
<p><strong>EDIT:</strong></p>
<p>To be clear, I'm not talking about database engines typically used for client-side storage, like SQLite, I'm talking about something that peeforms a role like VBA or C# which typically exist between the storage engine and the UI - but not those languages specifically, because I am arguing that they conceptualise things in ways different from SQL which is what creates the &quot;impedance mismatch&quot;.</p>

<language-design><language-ergonomics><nullability><programming-paradigms><data-types>
Why no "full-stack" SQL-like language?
CC BY-SA 4.0
2024-12-15T18:40:24.797Z
What kind of "front-end" functionality are you imagining? Would something like [Oracle Forms](https://en.m.wikipedia.org/wiki/Oracle_Forms) or [Oracle Apex](https://apex.oracle.com/en/) qualify?
IMSoP
CC BY-SA 4.0
2024-12-15T18:40:54.527Z
Basically, because SQL is not as expressive as other paradigms. You can resolve the "object-relational impedance mismatch" by adopting other paradigms besides OO without just writing everything in SQL, and if you do stick with OO then the "impedance mismatch" is often an acceptable cost.
kaya3
CC BY-SA 4.0
2024-12-15T18:51:24.227Z
Of course, the other answer is that a lot of applications *do* use client-side SQL databases. For example, Google Chrome stores your history, cookies etc. in an SQLite database; SQLite is also used by the rpm and Nix package managers on the client-side.
kaya3
CC BY-SA 4.0
2024-12-15T20:33:32.250Z
@kaya3, I'm not sure if I've been unclear somehow, but you're not following my thinking. I'm not talking about client-side storage using an SQL database engine, and I'm not talking about the SQL language itself being used, but about some unspecified language which, although presumably not SQL, nevertheless aligns sufficiently with SQL concepts that there is no "impedance mismatch". As I say, the absence of a null value within the domain of every type, is something that GP languages seem to lack intrinsically.
Steve
CC BY-SA 4.0
2024-12-15T20:37:35.780Z
@IMSoP, I don't know those applications first-hand, but from a brief read I suspect they did originally employ broadly the philosophy I'm talking about. But even they say they are now Java-based, and Java (to my knowledge) is not fully commensurate with SQL.
Steve
CC BY-SA 4.0
2024-12-15T20:46:27.377Z
Ruby has only a single type (it is dynamically typed, so from a type-theoretic standpoint, it is "uni-typed"), and it has a `nil` value which is a member of that type. Therefore, it fulfills your criteria of a general purpose language, often used on the client-side, which has a null value within the domain of every type. Is that what you are looking for?
Jörg W Mittag
CC BY-SA 4.0
2024-12-15T21:02:50.507Z
You might also be interested in some of the discussions on the Wiki, e.g., [ObjectRelationalImpedanceMismatch](https://wiki.c2.com/?ObjectRelationalImpedanceMismatch) and [ObjectRelationalImpedanceMismatchDoesNotExist](https://wiki.c2.com/?ObjectRelationalImpedanceMismatchDoesNotExist) as well as the many discussions that branch off of those two.
Jörg W Mittag
CC BY-SA 4.0
2024-12-15T21:46:12.060Z
@JörgWMittag, but SQL *isn't* "uni-typed". It actually has separate types, but with a null value in each. And it has a scalar/table distinction in its types. And type checking, and operators, etc. What you're describing about Ruby sounds much like VBA's Variant type. So VBA - which I mentioned - already fulfils the criteria as you're interpreting the criteria to be. I'm not sure you're following my thinking about the desirability of conceptual *exact alignment* with SQL in certain respects, including particularly it's type system in an exact way. (Not read the links yet btw).
Steve
CC BY-SA 4.0
2024-12-15T22:04:32.217Z
@JörgWMittag, regarding the second link about the non-existence of the impedance mismatch, I think that's primarily argued from the perspective of OO programmers whining about the relational model. I'm actually coming the other way, as a programmer who uses the relational model whining about OO languages!
Steve
CC BY-SA 4.0
2024-12-15T22:33:47.800Z
@Steve Well, as I said, there are lots of languages where there is no "object-relational impedance mismatch", as that is only a feature of OO languages. And SQL itself is a non-OO language which is used clientside.
kaya3
CC BY-SA 4.0
2024-12-15T22:37:09.793Z
@kaya3, as I say, I'm not sure you've followed my thinking. There's a *difference* between for example the type system of SQL and even old non-OO languages like C. I'm not sure why you're sustaining the point that SQL is used client-side, when I've already addressed your confusion in detail above.
Steve
CC BY-SA 4.0
2024-12-15T22:37:29.160Z
There is no difference between the type system of SQL and the type system of SQL, though. What about SQL makes it *not* an answer to your question? Yes, clearly, I'm not understanding something about what you want to know, which is why you need to clarify. You are asking *"about some unspecified language which, although presumably not SQL, nevertheless aligns sufficiently with SQL concepts that there is no "impedance mismatch""*. If it's only *presumably* not SQL, then you haven't ruled out SQL as a possible answer. And I don't see anything else which rules out SQL.
kaya3
CC BY-SA 4.0
2024-12-15T22:43:03.393Z
@kaya3, no but there's a blatantly obvious difference between the context of use, and I've given more than enough detail in my question to allow a sensible person to perceive that aspect. If you think I can replace, say, VBA with SQL - using SQL in places I would usually use VBA to show screens and set UI fields and display message boxes and react to button-presses and so on - then (to say the least) there's a considerable scope for use of SQL which I have not been previously aware.
Steve
CC BY-SA 4.0
2024-12-15T22:45:51.667Z
@Steve The only way for you to assert that you have given enough information for a sensible person to understand what you mean, is if you say everyone commenting or answering here is not sensible, because it seems *nobody* other than you understands what you are asking.
kaya3
CC BY-SA 4.0
2024-12-15T22:48:25.860Z
Nullable types are an extremely common feature of many languages, though implemented in many different ways; C# didn't belatedly add *nullable* types, it added *non-nullable* types (something which SQL also largely lacks). That's completely unrelated to the "object-relational impedance", and any other aspect of SQL.
IMSoP
CC BY-SA 4.0
2024-12-15T22:48:58.220Z
@kaya3, so what role did you see the example of Access and VBA, and my specific reference to the absence of nullable types in VBA despite being married to an SQL engine, playing in my question?
Steve
CC BY-SA 4.0
2024-12-15T22:51:41.867Z
@IMSoP, maybe I should have been more specific and said *nullable value types*? At any rate, even the types it has now are not quite full-square with the concept of the null value in SQL.
Steve
CC BY-SA 4.0
2024-12-15T23:00:49.137Z
@Steve No, nullable value types have been in C# from the beginning, as have implicitly nullable reference types; what's new is having a context where reference types can error if not *explicitly* nullable. Meanwhile just about every DBMS has a procedural language which builds on SQL, using the same type system as the main database (which may or may not match ANSI SQL). You really do need to be a bit clearer in the question about what you mean by "similar to SQL", and what a language needs to be able to do beyond that.
IMSoP
CC BY-SA 4.0
2024-12-15T23:38:48.950Z
@IMSoP, from what I can see from reading now and vaguely recollect, the last time I used C# on any regular basis (around v5.0), the language was in a condition where nullable types were virtually unusable, because they were explicit containment structures for the base value type with a bolted-on null field, not additional values in the domain of the basic type. Strings were treated differently. And they were all inconsistent with DBNull, as used by many library facilities. I'd practically forgotten the depth of this mess that prevailed at the time! (1/2)
Steve
CC BY-SA 4.0
2024-12-15T23:39:23.870Z
So in my view, this is not a point well-made that C# is a language whose type system aligned with SQL "from the beginning". I concede it has shaped up better in intervening years. There's at least a couple of reasons I'm discounting it though. First, C# as a system of programming has not been designed from the outset to align with SQL, and we're arguing here over a small and belated concession/easement, not showing full alignment. And second, the design does not proceed through to the front-end - for example, Winforms has no consistent built-in representation for nullable fields. (2/2)
Steve
CC BY-SA 4.0
2024-12-15T23:53:42.057Z
@IMSoP, and just to make my question emphatic, it's not "why can't various languages be pressed reluctantly into interacting with database engines", it is to ask why no language (that I'm aware of) appears to have been designed in which the initial, central, overarching goal was to avoid *any* inconsistency with SQL conceptualisations, right through to the UI layer. I'm not talking about strict syntax alignment with SQL either. (1/2)
Steve
CC BY-SA 4.0
2024-12-15T23:55:23.050Z
I'm talking about such things I mentioned in my question like table types, nullable types as they are in SQL, operators that handle nulls the same as SQL, and a reasonably suitable syntax reflecting the central design goal of conceptual alignment. The reason I think it's plausible to have expected such a thing by now, is the ubiquity of SQL engines, and the extent to which a great many bespoke applications fundamentally are client-server database applications. (2/2)
Steve
CC BY-SA 4.0
2024-12-16T02:32:32.033Z
For what it's worth, I don't find the intent of this question that hard to discern, although I agree that the title & introduction probably don't get to its point enough (and I don't think the present answer is a fair response to the question). However, there *are* a number of programming languages tightly coupled with databases that seem like they could be counterexamples - are you including/excluding things like MUMPS, Visual FoxPro, many so-called fourth-generation languages? Is the full relational model of ANSI SQL in particular the key aspect? Are non-commercial research systems included?
Michael Homer
CC BY-SA 4.0
2024-12-16T02:34:29.633Z
Looked at another way, would the answer "these do exist, they just aren't popular" be a frame challenge to the question, or a direct answer to it?
Michael Homer
CC BY-SA 4.0
2024-12-16T05:35:12.090Z
@MichaelHomer, I would say integration with SQL is a key aspect, because SQL is the thing which is widespread. I've heard of MUMPS, but (iirc) it's a medical system of ancient design and not at all SQL-based. FoxPro covers similar ground to MS Access, and (I believe) relates to SQL in a similar (equally deficient) manner. I'd consider it an answer if something did exist, and there was some theory about why it was unpopular. I'd just emphasise/clarify again that I'm not necessarily looking for something which is syntactically identical or even similar to SQL. (1/2)
Steve
CC BY-SA 4.0
2024-12-16T05:35:57.993Z
And I'm assuming there would be no significant query functionality in this client-side language. But there would have to be a native representation of tabular data sent from the server, the type system working the same so that every scalar value representable in SQL on the server side has a direct equivalent on the client (so that there is zero need for any translation code, or mental shift by the programmer), etc. (2/2)
Steve
CC BY-SA 4.0
2024-12-16T10:01:49.090Z
These comments have drifted a long way from what they're intended for, which is helping to improve the question. So, here's some concrete suggestions for an edit (as in rewording the whole thing, not adding more sections on the end): 1) drop the term "client-side", because you're using it in a non-standard/un-common way; 2) come up with a clear list of criteria that a language would need to match your idea, so we don't have endless No True Scotsman debates; 3) ask about why none have _succeeded_, rather than "devised"
IMSoP
CC BY-SA 4.0
2024-12-16T10:34:05.627Z
@IMSoP, (1) anyone familiar with a client-server database application will understand that term in context of the question, and you're not suggesting any better word; (2) I don't want to exhaustively define the criteria, instead I want you and others to get with the gist of question (or withdraw if your confusion is such that you cannot follow); (3) I'd be happy to know which had been devised but then failed - I've been believing that there hasn't been an attempt. Even allowing for my own muddled terminology, I'm shocked by the level of quarrelsomeness this question is provoking.
Steve
CC BY-SA 4.0
2024-12-16T11:00:11.837Z
@Steve Evidently people did *not* interpret "client-side" how you intended. That doesn't make you wrong, or them idiots; but it does mean that you might get more satisfying answers if you can think of a different way to express it. My suggestion was made in good faith: if you want to reduce the confusion and debate on the question, try clarifying it; if you're not interested in clarifying it, don't expect everyone to suddenly understand it.
IMSoP
CC BY-SA 4.0
2024-12-16T11:52:38.277Z
@IMSoP, the point of consulting a community of fellow experts is that they can get with it quickly and provide useful information, it's not supposed to be like confronting a gang of unruly philosophers. I admit "client-side" could be an ambiguous keyword on its own, but I'm less willing to admit that somebody who had read the full question could have attributed to it any meaning other than the one I intended. I'm not sure anyone was actually confused about that point. You commented in a way that suggested you understood my question. (1/2)
Steve
CC BY-SA 4.0
2024-12-16T11:56:27.157Z
And Jorg and Kaya responded in ways which suggested they fully got the meaning of "client-side", since they both referred (in responses which were bizarre for other reasons) to the common use of the SQLite storage engine on the client-side. Michael and Jerry have managed without quibble on the point. So what other term to use? If I had mistakenly overlooked a common and more precise term, I could fix. But you have no better suggestion, despite claiming my usage was unconventional, so why raise the complaint? (2/2)
Steve
CC BY-SA 4.0
2024-12-16T12:10:29.847Z
I've wondered about this issue for a long time too. (And understood it on the first read). Hope to see interesting answers
mousetail 'he-him'
CC BY-SA 4.0
2024-12-16T17:24:33.173Z
I'll lay my cards on the table: I have an idea what _might_ be being asked, and have some thoughts on how to answer that interpretation. But since anything I spend effort writing seems likely to receive a reaction of "you're missing the point", or worse that I'll be labelled "bizarre", "idiotic", and so on, I'm not particularly motivated to bother.
IMSoP
CC BY-SA 4.0
2024-12-16T18:26:29.007Z
I'm pretty sure I've seen something fitting the rough description built on XQuery, if you're willing to allow that much deviation from SQL (into general-purpose "query language" space).
Charles Duffy
CC BY-SA 4.0
2024-12-16T19:19:37.220Z
@IMSoP, I'd certainly be interested to hear your perspective; I'm hoping curiosity causes me no further regrets today. I'm still digesting what all this trouble meant.
Steve
CC BY-SA 4.0
2024-12-16T19:26:47.333Z
Computer science professor Ron van der Meyden summarized the various issues as: "The inconsistencies in the SQL standard mean that it is not possible to ascribe any intuitive logical semantics to the treatment of nulls in SQL."
Yakk - Adam Nevraumont
CC BY-SA 4.0
2024-12-16T20:19:14.630Z
@Yakk-AdamNevraumont, I've discussed these interesting points before with other people. Part of the problem is that the null value is not supposed to have one consistent intrinsic meaning, and it is in fact used for multiple different and unrelated purposes in SQL, for very good design reasons. The "treatment" reflects a well-balanced settlement of these different uses. Theoreticians and educators often don't like this complexity, but their objections to it are often naive.
Steve
CC BY-SA 4.0
2024-12-16T20:51:00.890Z
@Steve It was used for multiple different and unrelated purposes because people writing SQL where just bashing things together and picked something that sort of made sense. Treating quirky language choices as "very good design reasons" is language-hubris! More importantly, because NULL has no coherent meaning, extending the meaning of NULL to more cases than the eclectic mix of features SQL has has no coherent solution. How should NULL be handled when passed to an operation SQL lacks? You get to make it up, like the designers of SQL did at the time, and you'll get quirks.
Yakk - Adam Nevraumont
CC BY-SA 4.0
2024-12-16T20:59:09.960Z
What "all this trouble" means is that sometimes communication isn't easy, and what seems straightforward and clear to you isn't necessarily as clear to other people, so sometimes you have to put in a bit of effort to choose different words or clear up distracting side points, so that people aren't talking past each other. Repeatedly telling people that they *should* understand, and are idiots if they don't, rarely helps.
IMSoP
CC BY-SA 4.0
2024-12-16T21:42:04.317Z
Your question puts me in mind of [data oriented design](https://en.wikipedia.org/wiki/Data-oriented_design), which often takes the form of "table based" approach to programming. I don't know if there's a mainstream DOD-based programming language yet (akin to Java/Smalltalk for OOP), but I'd imagine someone is researching it.
R.M.
CC BY-SA 4.0
2024-12-16T21:44:44.087Z
@IMSoP, well one thing I've taken on board is that when I'm asking a question - as a computing expert with considerable general knowledge already - it is almost certain to be something highly obscure, whose implied details would probably be obvious only to others with comparably deep expertise, and I'm implicitly looking for someone who already knows more than me (including whatever I can already gain from research). It might not always be obvious to others how little they know. (1/2)
Steve
CC BY-SA 4.0
2024-12-16T21:45:02.313Z
I noticed contemporaneously how quickly I was getting riled in the role of a questioner, compared to my relatively long patience in my usual role as an answerer. I think the root of this - besides my own towering arrogance - is that I'm not particularly well-known here and I obviously hadn't made my grade of expertise sufficiently clear, which perhaps would have produced more pause for thought before some of the less useful replies were plied. (2/2)
Steve
CC BY-SA 4.0
2024-12-16T21:54:11.153Z
@R.M., yes it's a good point. What they're calling "data-oriented design" is basically just a return to the status quo ante before OOP, of structured programming, procedural code, and data separate from algorithms.
Steve
CC BY-SA 4.0
2024-12-16T23:57:54.730Z
@K.A.Buhr, Haskell? Good heavens, the best cure since they gave electroshock to Hemingway.
Steve
CC BY-SA 4.0
2024-12-17T03:45:34.657Z
I think you're looking for Informix-4GL, Natural/ADABAS, 4Js Genero, OpenEdgge ABL, OpenROAD, etc.
user71659
CC BY-SA 4.0
2024-12-17T14:01:47.777Z
@user71659, yes I'd say (as a non-user who has quickly looked them up...) those are potentially languages meeting the criteria. Have you any insight to offer as to why they aren't as popular as SQL itself?
Steve
CC BY-SA 4.0
2024-12-17T22:07:56.837Z
PowerBuilder is an SQL based language for building thick client front ends off an SQL database. I would not recommend - having to read it in order to port away is bad enough.
Rich
CC BY-SA 4.0
2024-12-17T22:21:59.457Z
@Steve I suspect the issue is developers don't want to learn a language and build skills with a narrow application space, coupled with the fact that there's no clear winner and that ends up in a proprietary stack.
user71659
CC BY-SA 4.0
2024-12-17T23:17:32.390Z
@user71659, that aligns with my suspicions.
Steve
CC BY-SA 4.0
2024-12-18T00:59:41.073Z
@Steve "Have you any insight to offer as to why they aren't as popular as SQL itself? ". Mostly because while _very_ well suited to doing SQL, they're not very well suited to doing anything else. I've worked a lot with Informix 4gl and Genero over the years, and they are _not_ fun languages to work with, despite the many improvements Genero has made over its predecessor.
Simon Geard
CC BY-SA 4.0
2024-12-18T01:03:02.330Z
As a general rule, you can't hire developers who already know those languages... and if you ask the developers you _do_ hire to work on them, they tend to resign because they'd rather work on something that's more fun and/or looks better on their CV.
Simon Geard
CC BY-SA 4.0
2024-12-18T11:05:14.413Z
Regarding people (mis)understanding your question; your texts are very slow to come to the point, and use a lot of words to say very little. Some concrete examples: _"... it is regarded as extremely well-adapted for its various and widespread uses in data processing."_ - All weasel words and fluff saying nothing. - _"It has various characteristics ..."_ - Of course it does. _Everything_ does. - _"now that's a pugilistic manner of entrance!"_ - Those are your first words in one of your comments here. What even? - You may want to consider improving the clarity of your writing.
marcelm
CC BY-SA 4.0
2024-12-18T11:16:41.657Z
@marcelm, at times I'm tempted to think I use too many words, but then the extremity of misunderstanding suggests I've used far too few. The purpose of certain sentences is to show what thinking has produced the question (such as about the well-adaptation of SQL, showing clearly that I regard it as the non-negotiable in the overall stack of languages) - I'm happy to sustain an explicit frame challenge, but I don't want people just talking at cross purposes to me, although they have anyway. Hopefully that explains why I regarded the opener of J Mini's answer as fighting talk, said playfully.
Steve
CC BY-SA 4.0
2024-12-18T12:22:18.717Z
@Steve I disagree with the premise that more words correlates with more clarity. In fact, I find that often the opposite is true. The key is to ensure every word is important to the essence of your communication.
marcelm
CC BY-SA 4.0
2024-12-18T13:58:13.460Z
@marcelm, I agree, and I've explained that an example you adduced of fluff actually had a specifically intended purpose when I wrote it. I'm not immune to rambling and wordy sentences, but many of you appear to be just complaining that the question is not sufficiently simple or reducible to a pithy one-liner, when there is no reason it ought to be. As for lack of clarity, I'm not at all satisfied that those lodging complaints have first attempted reasonable interpretation. (1/2)
Steve
CC BY-SA 4.0
2024-12-18T13:58:25.520Z
"It has various characteristics..." was used to introduce a list of *relevant* characteristics, and I recall that I first wrote that sentence without the parenthetical remark, then added the parenthesis to emphasise that I was listing an intentional selection. I could reword it to maybe achieve greater artfulness as a sentence, but I struggle to see how I could actually purge words from the overall count, or that it is particularly infelicitous (let alone so inept as to be worthy of your main focus). (2/2)
Steve
CC BY-SA 4.0
2024-12-18T16:59:09.357Z
@steve You write, _"I could reword it to maybe achieve greater artfulness as a sentence, but I struggle to see how I could actually purge words from the overall count, or that it is particularly infelicitous (let alone so inept as to be worthy of your main focus)."_ (244 chars) - I might say, _"I could improve it, but I fail to see how to make it shorter, or that it even needs to be."_ (90 chars). That's the same message, in about a third of the characters and much clearer. Your wordiness makes it hard for me to parse out what you actually mean, and I suspect I'm not the only one.
marcelm
CC BY-SA 4.0
2024-12-18T17:40:50.843Z
@marcelm, nicely done, although it is less specific. I meant the available margin of improvement shows you to be a nit-picker whose attention is misdirected, and I didn't want that to be only implicit.
Steve
CC BY-SA 4.0
2024-12-18T20:57:37.047Z
While the responses to this question seem to have gone irretrievably off-track, I do think that there is a useful core question something along the lines of "Why was the approach of the group of general-purpose 'fourth-generation languages' that were tightly coupled with SQL not more widely adopted?". The body would identify the aspects you're most interested in, which I think are particularly creating user interfaces and the relational data model, and the question wouldn't start from the premise that these never existed. This would be a separate, shorter question about design characteristics.
Michael Homer
CC BY-SA 4.0
2024-12-18T22:42:49.867Z
@MichaelHomer, I couldn't cope with another question!
Steve
CC BY-SA 4.0
2024-12-19T13:40:36.540Z
Those languages exist(ed), they used to be called 4GL languages. Oracle has tools to write full apps with only SQL and PL/SQL (Forms, Reports, APEX). They use mostly the same data types that the database uses. Obviously they're all closed, proprietary and little known tools unless you're already an Oracle customer. And they may look really really reall old-fashioned (Forms and Reports mainly; APEX is usable in modern contexts but still a little hard to version-control).
marcus
CC BY-SA 4.0
2024-12-19T17:38:06.450Z
@Steve There seemed to be a lot of confusion as to what you actually meant with your question. All I tried to do is offer more concrete examples of issues I noticed, in the hope to better help you fix this confusion. Do with my feedback what you will, I'm bowing out of this conversation.
marcelm
CC BY-SA 4.0
2024-12-26T10:19:55.213Z
I'm interpreting this question very differently than the existing answers do. The question I see is: why doesn't the approach SQL was based on get extended to the GUI? I had the same question in the 1990s and I was working on a language to do this, but never finished it. I wanted to explain the research question and my approach to answering it in an answer, but that is impossible when the question is closed. What to do now?
reinierpost
CC BY-SA 4.0
2024-12-26T12:32:16.480Z
@MichaelHomer, is there any chance we can get this question reopened in light of the above commenter's intentions?
Steve
CC BY-SA 4.0
2024-12-26T18:58:16.283Z
@reinierpost I've reopened and unprotected the question so that you can post an answer now.
Michael Homer
CC BY-SA 4.0
2024-12-26T19:20:47.007Z
@MichaelHomer: ❤️️
reinierpost
<h2>LINQ, and null coalescing operator <code>??</code></h2>
<blockquote>
<p>What I'm confused about, and which brings me to my question, is why no system of programming has ever been devised that simply attempts to run with SQL concepts (like listed above) throughout the design, right through to the front-end, thus resolving the notorious &quot;impedance mismatch&quot;?</p>
</blockquote>
<p>You may be interested in <a href="https://en.wikipedia.org/wiki/Language_Integrated_Query" rel="nofollow noreferrer">Language Integrated Query</a>. Complemented with the null coalescing operator <code>??</code>, it appears to tick all the boxes.</p>
<blockquote>
<ul>
<li>tables as first-class types with associated operators (Join, aggregate, etc.)</li>
</ul>
</blockquote>
<p>Tables, or to be precise, any <code>IEnumerable</code> entity, can be selected, matched, filtered, and have functions applied over, inclusive using near identical SQL <em>syntax</em>. The example below is valid C# code!</p>
<pre class="lang-csharp prettyprint-override"><code>var artists = from a in context.Artists
              where a.Name.StartsWith(&quot;A&quot;)
              orderby a.Name
              select a
</code></pre>
<blockquote>
<ul>
<li>every scalar type has a first-class null value</li>
</ul>
</blockquote>
<p><em>Everything</em> in C# is now nullable.</p>
<blockquote>
<ul>
<li>a particular (and justifiable) system of how operators handle null values.</li>
</ul>
</blockquote>
<p>Testing for <code>== null</code> and skipping nulls with <code>??</code> are both possible, both in class and syntax forms.</p>

CC BY-SA 4.0
2024-12-16T21:10:17.330Z
2024-12-21T16:56:32.133Z
2024-12-21T16:56:32.133Z
André L F S Bacci
CC BY-SA 4.0
2024-12-17T01:37:45.557Z
^^^ This. I'm surprised no one else has mentioned LINQ yet.
PC Luddite
CC BY-SA 4.0
2024-12-17T01:37:49.007Z
Although based on the comments to the question, I imagine this doesn't fit the narrow criteria of the OP.
PC Luddite
CC BY-SA 4.0
2024-12-18T09:08:42.427Z
I think this answer supports the answer which suggests the problem is that SQL has serious flaws and the comment about it being very ad-hoc. I found LINQ a bit difficult to wrap my head around because it simultaneously seems to turn SQL inside out (e.g. FROM comes first, SELECT comes last) while also doing little to fix the ad-hocness.
Michael
CC BY-SA 4.0
2024-12-18T16:07:34.033Z
@Michael, putting the from-clause first is actually one thing to which I've never heard an objection! When I write SQL, I often start with a boilerplate `SELECT * FROM` and then proceed to write the actual query, going back to modify the select-clause once the relevant fields are in scope. The confusing thing about Linq - I used to find - was predominantly related to the more complicated data shapes (and their typing) it was capable of working with, inspecting intermediate results, or reasoning about the moment and effects of evaluation.
Steve
CC BY-SA 4.0
2024-12-19T18:49:34.850Z
@Steve Oh I agree, when you think about it, it does make sense that FROM comes first, in fact the order in which SELECT specifies things is a bit out of order from how the query actually works.  I'm not sure if Linq entirely fixes that, but it does take a bit of unlearning.
Michael
<p>You ask,</p>
<blockquote>
<p>why no system of programming has ever been devised that simply attempts to run with SQL concepts [...] throughout the design, right through to the front-end, thus resolving the notorious &quot;impedance mismatch&quot;?</p>
</blockquote>
<p>Fundamentally, I'd attribute that to SQL having only one form of data organization: the table.  That serves well enough (though by no means flawlessly) for data storage, but tabular organization is simply not natural for a lot of the data that applications work with.  This is precisely the nature of the &quot;impedance mismatch&quot; in the first place.</p>
<p>That doesn't mean that table data types and relational algebra operators couldn't be provided (and I've occasionally seen them in libraries), but as soon as you give a language any other kind of aggregate data type, you introduce the mismatch that you ask about, between tabular data types and other aggregates.  You just get it directly in the application language instead of between the application language and the data access language.</p>
<p>By and large, people making a language choice choose a language that they expect to be one or more of convenient, comfortable, effective, productive, performant, or cost-efficient.  Maybe I've missed one or two.  A table-oriented language isn't any of that for much of any problem other than storage.  Instead, it ticks pretty much every box to have more flexibility. That doesn't mean that such a language couldn't get the job done, just that no one has any motivation to create the language.</p>
<p>SQL is as successful as it is largely because it doesn't have to provide an end-to-end solution.  Its success in its niche should not be mistaken as a sign that it, or something like it, would be well adapted to a broader role.</p>

CC BY-SA 4.0
2024-12-16T22:26:24.877Z
2024-12-16T23:02:23.850Z
2024-12-16T23:02:23.850Z
John Bollinger
CC BY-SA 4.0
2024-12-16T22:59:41.380Z
To summarize: SQL is a domain-specific language.
RonJohn
CC BY-SA 4.0
2024-12-16T23:04:20.207Z
The tabular model in SQL is actually key to it's flexibility and queryability (so to speak), not restrictiveness. There's no *single* other data structure that is quite as flexible for automatic processing, including ease of mechanical transformation into more complicated shapes (as is often done for final presentation). I'm not saying this mystery new language would have to confine itself only to tables of course, I'm just remarking on the question of whether tables are limiting or not.
Steve
CC BY-SA 4.0
2024-12-16T23:15:39.683Z
@RonJohn, yes but the "domain" here actually covers the majority of the purpose for which organisations use computers, which is automatic data processing. Whilst I understand the concept of a "general purpose" programming language as typically meaning you can write an operating system as much as a business application, the majority of languages and frameworks in use today are actually "domain specific" by that old criteria. So I wouldn't be tempted to think that SQL is doing something narrow - it's a *highly general* language for a domain of considerable size and importance.
Steve
CC BY-SA 4.0
2024-12-17T04:14:32.497Z
@Steve, I think I saw some of the "everything's an object" guys looking for you.  But seriously, the tabular model is flexible in the sense of being able to model data, but not more so than Java or C++ classes, for example, since classes can implement tables.  The tabular model is, I agree, central to querying, but querying is not the be all and end all of application programming.  Data representation matters, programming paradigm matters, and a few other things that tables are not flexible about sometimes matter too.
John Bollinger
CC BY-SA 4.0
2024-12-17T13:07:35.977Z
@Steve You seem to be ignoring efficiency here. You _could_ implement almost any data structure you can think of in SQL using tables and queries. But for almost anything other than basic mapping types and and classes that only store data, it’s horribly inefficient. There’s a very good reason that things like time-series and graph databases exist, the data structures they represent can’t be efficiently implemented using SQL, and _efficiency is important_.
Austin Hemmelgarn
CC BY-SA 4.0
2024-12-17T15:10:18.500Z
@JohnBollinger, I agree with your points. I'm not thinking of a language that restricts itself to only a tabular model. But since SQL does work exclusively on a tabular model, the mystery language should have a particularly efficient manner of handling tables (which will often be going to or coming from an SQL engine).
Steve
CC BY-SA 4.0
2024-12-17T15:19:45.137Z
@AustinHemmelgarn, yes, I wouldn't want to argue that SQL engines must be best for every conceivable data processing application, although I'm not sure to what extent the problems you identify are due to the tabular conceptual model in SQL (as opposed to the particular design of the storage subsystems of mainstream SQL engines). Rather, I'm proceeding from the premise that their popularity and ubiquity in practice - which has grown despite years of considerable opposition amongst the punditry, whilst the proposed alternatives all flashed in the pan - is proof of the general usefulness.
Steve
CC BY-SA 4.0
2024-12-17T16:42:54.510Z
@Steve Proposed alternatives all flashed in the pan? You must not have heard of the numerous TSDB, document database, graph database, and KV store projects that are doing very well and seeing plenty of widespread usage. Just off the top of my head, MongoDB, Riak, Neo4j and RocksDB are at least as big as the major SQL platforms (other than SQLite, but that is ubiquitous for reasons other than it’s usage of SQL), and there are plenty of others out there as well.
Austin Hemmelgarn
CC BY-SA 4.0
2024-12-17T16:46:11.637Z
@Steve And as far as the difference between the SQL engine and the data model itself, the issue is more in the data model and the storage formats. Graph structures _cannot_ be handled efficiently within the standard SQL data model due to scaling issues (the data model itself inherently turns things into _n²_ or worse efficiency in either time or space based on the number of nodes or edges involved), and efficient TSDB usage requires different storage than what SQL systems use (and optimizing for TSDB usage there will make ‘regular’ SQL usage less efficient, because they need different things).
Austin Hemmelgarn
CC BY-SA 4.0
2024-12-17T18:45:30.283Z
@Steve, you write "*I'm not thinking of a language that restricts itself to only a tabular model*", but this answer already speaks to that.  The kind of language you are now talking about does not resolve the impedance problem, but rather internalizes it. Tables are good for storage.  Other kinds of data structures tend to be preferred for processing. The mismatch hits when you want to move between those.
John Bollinger
CC BY-SA 4.0
2024-12-17T21:02:51.863Z
@JohnBollinger, I think the "impedance mismatch" is caused not just by the change in data structure per se, but by the fact that it has to be done right at the interface between two incommensurate languages, rather than inside one language that is designed to neatly handle SQL result sets on their own terms (as well as designed to handle any further transformation that may be desired).
Steve
CC BY-SA 4.0
2024-12-17T21:40:03.553Z
@Steve, this answer already claims otherwise.  And your assertion leads me to think that you may have a slightly incorrect idea about the nature of the "impedance mismatch".  The mismatch is that whereas tabular representation is well suited for storage, it is not especially well suited for many kinds of processing operations, while those representations that are well suited for processing are not so well suited for storage.  Moving the point where you change between those does not reduce that.  Neither does making such changes easier, if indeed you can.
John Bollinger
CC BY-SA 4.0
2024-12-17T22:46:02.203Z
@JohnBollinger, although I'm not being exhaustive about all its aspects, the impedance mismatch is about a clash of conceptualisations, in my view. The tabular model partly emerged because more complicated structures (which had already been tried before the SQL era), whilst they may be highly effective for certain queries, are highly ineffective for others - that is, no single more-complicated form copes better in general than tables. More complicated forms are also more difficult to express queries for (that is, specify the desired processing upon the data). (1/2)
Steve
CC BY-SA 4.0
2024-12-17T22:47:38.053Z
The reason I'm suggesting "moving the translation point" fully into the language, is because by definition the language must then incorporate both conceptualisations and reconcile them somehow in order to facilitate translation of the data between each. Whereas when the translation is done immediately at the point where the data first enters the scope of the client-side language, then you have an unholy situation where the translation awkwardness for the language user is mandatory rather than optional, and where the language designer tends not to take responsibility for that awkwardness. (2/2)
Steve
CC BY-SA 4.0
2024-12-17T23:03:43.693Z
At this point, @Steve, it's clear that we have moved off the question posed, which supposes that one could *resolve* the mismatch by use of a general-purpose language that was substantially SQL-like.  Maybe one could, but, again, no one wants to do application programming in that language.  Now you're suggesting a language that *internalizes* the mismatch and moderates its effects, which is different.  I think if it were as easy as that then it wouldn't need to have a language built around it in the first place, and it probably would have been done already. [...]
John Bollinger
CC BY-SA 4.0
2024-12-17T23:04:19.650Z
[...] But again, that's a different question, not the one to which this answer is directed.
John Bollinger
CC BY-SA 4.0
2024-12-17T23:23:49.503Z
@JohnBollinger, I suppose I'm playing two hands here. I think a client-side language that just runs with certain SQL concepts, would work better for many applications. But I'm also provisionally accepting your point that maybe something about existing client-side languages works better in other cases, and therefore a translation would still be necessary, rather than simply aligning fully to SQL concepts. Anyway I'll leave it there, I suppose, rather than drag you any further into the complications.
Steve
CC BY-SA 4.0
2024-12-18T15:09:56.653Z
One problem tables (multisets) are well suited for other than storage is distributed compute. Apache Spark RDDs are basically another form of relational algebra, but with an object encoding built on top. The non-ordered set properties make distributing the work by partitioning the tuples really simple.
user1937198
<h3>Because SQL Has Serious Flaws</h3>
<p>SQL is an <em>acceptable</em> language for doing CRUD on a database. I cannot think of anyone, not even professional DBAs, who would consider SQL a good foundation to build upon. Some major faults come immediately to mind:</p>
<ul>
<li>SQL is not an extensible language. I'm not making some Lisp-like argument about how you cannot metaprogram it. My argument is far nastier: most SQLs <em>don't even have libraries</em>.</li>
<li>Unless you're treating SQL like an assembly language and doing performance tuning, <a href="https://www.linqpad.net/WhyLINQBeatsSQL.aspx" rel="noreferrer">ORMs beat it in every aspect</a>. I cannot name <em>anything</em> that SQL does better than an ORM.</li>
<li>Unit testing, one of the cornerstones of modern application development, is extraordinarily difficult in SQL. The language is so limited that a common SQL Server solution is to <a href="https://tsqlt.org/" rel="noreferrer">use C# to hack it in</a>.</li>
</ul>
<p>None of what I'm saying here is new. You can find lengthy critiques of SQL from <a href="https://dl.acm.org/doi/pdf/10.1145/984549.984551" rel="noreferrer">as far back as the 1980s</a>. Attempts to improve on it, such as <a href="https://www.edgedb.com/blog/we-can-do-better-than-sql" rel="noreferrer">this one</a>, tend not to catch on <a href="https://www.brentozar.com/archive/2024/06/the-sql-language-is-just-a-hurdle-you-gotta-overcome/" rel="noreferrer">because everything is expected to be compatible with SQL</a>.</p>
<p>Your question of why we don't have an application language that solves the impedance mismatch problem or uses <code>NULL</code>s just like SQL does (i.e. <a href="https://sqlperformance.com/2020/01/t-sql-queries/null-complexities-part-2" rel="noreferrer">inconsistently</a>) is entirely sensible, but the question of why we haven't build on SQL to make an application language is much more easily answered by this. SQL just isn't very good at anything other than querying. If you want an application language that handles SQL-like things, then you just use an ORM and you get something <em>better</em> than SQL.</p>

CC BY-SA 4.0
2024-12-16T19:50:33.593Z
2024-12-16T20:37:59.743Z
2024-12-16T20:37:59.743Z
J. Mini
CC BY-SA 4.0
2024-12-16T20:32:42.757Z
*"Because SQL Has Serious Flaws"* - now that's a pugilistic manner of entrance! I do take the point that not even SQL practitioners regard it as perfect, and that's why I'm not simply asking why we don't generalise the use of SQL, but rather (instead of ORMs) why we don't have languages that maintain certain key conceptualisations that SQL uses. A lot of SQL is ultimately designed for things like efficient algebraic optimisation, concurrent execution with transactional consistency, and online hot-patching, and in these areas it would leave other languages for dust.
Steve
CC BY-SA 4.0
2024-12-16T21:14:51.840Z
One of the things that makes SQL very hard to extend, or generalise, is that its syntax and semantics are both very *ad hoc*. Functions like `EXTRACT` use context-sensitive keywords that don't follow any general grammar, and each major statement has clearly been designed top-down, rather than composed from reusable components. PostgreSQL is among the most flexible implementations, and it's probably no coincidence that it started as a *non-SQL* relational database (Ingres -> Postgres -> PostgreSQL)
IMSoP
CC BY-SA 4.0
2024-12-16T21:29:38.910Z
@Steve I think the fact that SQL is a) dominant, and b) not very well designed *is* a good reason why no such language has succeeded: if the selling point is "the procedural/functional/imperative/OO part may look weird, but the query part is familiar", you have to start with SQL and botch yet more onto it. The alternative is to have a language where programmers have to learn a new query language *and* a new programming paradigm for the rest of their program. If there was a thriving ecosystem of query languages, there might be more chance that one would evolve into something general-purpose.
IMSoP
CC BY-SA 4.0
2024-12-16T22:38:28.323Z
@IMSoP, there's definitely lots of non-generality in SQL. What I imagine for this client-side language is not something that reproduces the syntax of SQL (or its oddities) nor really the query functionality. I keep coming back to this touchstone about the representation of null in SQL. For example, in SQL, nullable types are not composites of non-nullable types and an additional flag - they are not sum types. The null value exists directly in the domain of each data type. There are in fact no non-nullable data types - there only constraints that stop nulls being stored in certain cases. (1/2)
Steve
CC BY-SA 4.0
2024-12-16T22:39:32.323Z
These subtleties are actually quite crucial to how SQL works systematically, and they seem often to be misunderstood. I don't want to reduce the whole question to nulls, but I would expect this mystery/new language to at least have *exactly* these data types available in which *no* structural distinction is made between the ordinary values of each data type and the null value which each data type also has. But the syntax can certainly vary from SQL. (2/2)
Steve
CC BY-SA 4.0
2024-12-16T22:46:27.983Z
ORMs better than SQL?  Hah!!  That crap that ORMs generate has to be seen to be believed.
RonJohn
CC BY-SA 4.0
2024-12-16T23:13:13.737Z
@RonJohn I know that they generate bad SQL. That's why I specifically mentioned performance. _I have seen their evil_.
J. Mini
CC BY-SA 4.0
2024-12-17T17:58:48.523Z
That edgeDB page also has a good [critique of SQL](https://www.edgedb.com/blog/we-can-do-better-than-sql#critique-of-sql) on it which was very much worth reading.
JimmyJames
CC BY-SA 4.0
2024-12-17T22:15:19.393Z
@JimmyJames, it was worth a read, but I'm not sure I agree with that document at all. They start off with a point about the syntactic similarity of "table and scalar expressions", but in fact the examples are both table expressions. What they identify is the implicit cast of a one-row, one-column table to a scalar - what APL calls "floating". The only heading I was minded to agree with was "lack of consistency". I'm not sure they reveal sufficient familiarity with SQL concepts to be proposing innovations to it.
Steve
CC BY-SA 4.0
2024-12-18T07:23:43.720Z
please explain in what way ORMs are better than SQL.
Jasen
CC BY-SA 4.0
2024-12-18T10:35:04.267Z
@Steve: " The null value exists directly in the domain of each data type. There are in fact no non-nullable data types" - why would you want to design a language that bakes in the [billion-dollar mistake](https://en.wikipedia.org/wiki/Tony_Hoare)?
psmears
CC BY-SA 4.0
2024-12-18T11:26:52.010Z
@psmears, because Tony Hoare was talking about the null memory pointer. It's a fairly different concept, and has no analogy in SQL which has always been completely memory safe in that regard. I can't emphasise enough that the homonym does not imply conceptual similarity.
Steve
CC BY-SA 4.0
2024-12-18T12:09:15.103Z
@Steve: The concepts _are_ similar, though; in both cases you're adding a new value to a domain, that needs careful special-case handling. It is not obvious what the right semantics for NULL are - what's correct depends on the situation (ask three programmers how NULLs should behave and you'll get five different answers!). NULL and NULL-like constructs have their uses, but forcing _all_ datatypes to include them invites more harm than good. That's why modern languages don't tend to do that.
psmears
CC BY-SA 4.0
2024-12-18T13:12:18.153Z
@psmears, they really aren't that similar. The perceived similarity is in the general purpose of use, but it's like saying that because we can eat either insects or pigs for food, therefore insects and pigs are the same things. Indeed the analogy you are perceiving is between foreign keys and pointers. A foreign key can be null in SQL, but null is not used exclusively in the context of foreign keys but has substantial other roles too. And unlike the null pointer which is an alias for the zero pointer, the SQL null is not an alias for any other value. (1/2)
Steve
CC BY-SA 4.0
2024-12-18T13:13:22.430Z
Most of the quibbles about null - some expert ones are justified, but many I've heard over time are naive - relate to how the various operators handle nulls, not to the existence of the null values themselves in the domains of all the data types, which is fundamental to SQL and without which it cannot work anything like as we know it. Usually, the issue with null-handling is that SQL takes one approach in all cases when there should be more than one approach for different cases. I can't immediately think of any example where SQL just takes the wrong approach. Try me if you like. (2/2)
Steve
CC BY-SA 4.0
2024-12-19T07:02:59.560Z
@Steve The issue with how SQL handles null is that some parts of the language treat is as "applicable, but unknown" but others treat it as "inapplicable". For example, nullable bits ought to have a four-value logic but SQL has them as a three-value system. My answer links to one article about this.
J. Mini
CC BY-SA 4.0
2024-12-19T08:30:19.627Z
@J.Mini, that was Codd's later view, that there should be 4VL, but the problem with distinguishing missing and inapplicable, is that the prior question of whether the value is missing or inapplicable can also be unknown. See for example what I wrote on the topic here: https://langdev.stackexchange.com/a/3418/1411. In my view, this approach to improvement is misguided. The actual role of null is as a kind of ground state, a dustbin for *all* the reasons why the data cannot take on a more specific value. (1/2)
Steve
CC BY-SA 4.0
2024-12-19T08:30:24.883Z
The 4VL approach seemingly attempts to make null fully-specific in its meaning again, denying the dustbin/groundstate role, the very existence of such a role being what contributes the majority of its usefulness (unless you go to 5VL, and have missing, inapplicable, and groundstate). My experience is that missing/inapplicable don't tend to need distinct *handling* (I can recall occasional cases where the distinction was useful to propagate to the user, but not where the automatic handling differed). (2/2)
Steve
CC BY-SA 4.0
2024-12-19T11:13:24.260Z
@J.Mini, just picking up another track here, you link to a unit testing framework for SQL. The problem with unit tests is that my experience of working even with very weak SQL developers is that their main faults manifest in three areas: (1) fundamental analysis faults where they haven't grasped what a correct result looks like even in principle; (2) concurrency faults/performance emergencies which only system/integration tests could detect; (3) carelessness in general. Although not zero, the number of faults where unit testing seems likely to catch, are a small minority.
Steve
CC BY-SA 4.0
2024-12-19T11:22:40.443Z
@Hobbamok, the problem with ORMs is that the vast majority of applications aren't confined just to the simple cases they handle well. If you have a legion of low-grade application programmers plus a few expert SQL developers to take over all the hard cases or performance hotspots, then it might at least facilitate a division of labour, but for any developers who have to handle the whole stack (including any maintenance developers/DBAs later) it introduces an additional layer of complications.
Steve
CC BY-SA 4.0
2024-12-19T11:48:24.330Z
@Hobbamok, I'm just saying most multi-user applications simply *don't* fit in that simple case. It doesn't invalidate your point, it's just emphatic about those "shitshow" cases, and about the division of labour aspect (it's more sensible when there's a division of labour and releases some developers from needing any SQL skills, than for full-stack developers who still need all the skills).
Steve
CC BY-SA 4.0
2024-12-19T15:04:49.040Z
@Hobbamok, when I said "multi-user", I didn't have a specific threshold in mind but it was more like, you know, 20 or 50 concurrent sessions - far from multi-millions!
Steve
CC BY-SA 4.0
2024-12-20T20:28:36.040Z
The citation for ORMs beating SQL is about how nice their syntax is. This question, on the other hand, has nothing to do with syntax but asks generally about languages with SQL-like semantics, a category that includes LINQ. LINQ has the same thing as SQL, that it only applies to data queries and cannot be used to write a whole application front-to-back.
Criticize SE actions means ban
CC BY-SA 4.0
2024-12-26T06:29:52.780Z
@Hobbamok, I find that quite remarkable really, that a company can commission a database application for 20 or 50 *concurrent* users, and not have anyone involved in the development with real database skills and experience (probably explains that EAV disaster I once saw!). I did concede ORMs have a valid point - to enable a division of labour between those who can with SQL, and those who cannot.
Steve
CC BY-SA 4.0
2024-12-26T10:17:34.727Z
@MisterMiyagi, I concede your point, but the primary use-case for client-server SQL engines, is for many users to be concurrently reading and altering the same data (which is usually has significant volume and variety). Concurrency and volume, those are why you have to use SQL engines and not any other. If the application has neither concurrency (in the relevant sense) nor volume, and if you have nobody on the team who is a competent SQL developer, then it more raises the question why it's perceived as desirable to use an SQL engine without a single SQL *engineer*? Then an ORM layer on top?
Steve
CC BY-SA 4.0
2024-12-26T15:38:49.387Z
@MisterMiyagi, I agree and I know this, but question is, why would you use SQLite, if all you have on your team is people who *don't* know any SQL? For a single-user business application, with DotNet for example, you could just serialise the entire in-memory object graph - probably without writing more than a few lines to set up a built-in serialiser. I just don't understand how you get from "we're a team that doesn't know any SQL now, and won't learn!" to "we'll use SQLite plus an ORM framework for local storage in our simple single-user application!" 
Steve
CC BY-SA 4.0
2024-12-26T18:49:27.263Z
@MisterMiyagi, *I'm* not arguing against SQLite. You don't need to explain these advantages to me - I'm a fair expert SQL and pro-SQL. You need to explain why a team which knows little or nothing about SQL, can and do nevertheless appreciate all these advantages of using the relational model and an SQL engine? I find it relatively implausible that a team who have no SQL skills themselves and only simple (single-user, low-volume) storage needs, would nevertheless adopt these supremely positive attitudes towards the use of SQL engines and ORMs? Why do they need indexing if it's low-volume?
Steve
CC BY-SA 4.0
2024-12-26T20:10:01.467Z
@MisterMiyagi, it's one thing to know the SQL syntax and dislike it. I can follow the logic about how a person who knows SQL and understands relational databases, but vehemently dislikes SQL, can learn an ORM and use that - and even if I'm saying this is less efficient, his hatred will drive him through, and he can still revert to the SQL he knows and hates whenever the ORM reaches its limits. But I'm still not convinced by your last assertion - you're not talking about people who know and hate SQL, you're talking about people who don't even know how SQL works in the first place.
Steve
CC BY-SA 4.0
2024-12-26T23:00:56.047Z
@MisterMiyagi, I understand the distinction you're making, but in practice I still find it very doubtful that someone would have a good working grasp of SQL engines without knowing any SQL! I struggle to imagine the motivation or the professional circumstances of someone who acquires some kind of working conceptual knowledge of SQL engines, and accepts they are a good useful technology, yet manages to dodge any encounter with the SQL lingua franca of these engines! I'm not even sure why you're trying to establish the existence of this person - do you actually know a real person like this?
Steve
CC BY-SA 4.0
2024-12-26T23:13:40.253Z
I think the point is that SQL engines are not the only kind of database handler out there.
Someone
CC BY-SA 4.0
2024-12-27T08:56:40.847Z
@Someone, why would we be discussing ORMs if the point was about non-SQL databases?
Steve
CC BY-SA 4.0
2024-12-27T09:47:16.703Z
@MisterMiyagi, I began by talking about *normal* database applications. I had two points: (a) that ORMs don't enable strong SQL skills to be eliminated from the development team when writing normal database applications; and (b) ORMs themselves inflict additional overheads (they're primarily designed to facilitate division of labour and skills specialisation, not to boost the productivity of a full-stack developer, the latter of whom will pay the overhead but will not benefit from any division or specialisation).
Steve
<h1>Disclaimer</h1>
<p>The question asked is specifically <em>why</em> this hasn't happened. Not about whether it's a good idea, helpful, etc. So quite a bit of what I have to say is related solely to why things are the way they are, with no comment about other questions that could be seen as related.</p>
<h1>Answer</h1>
<p>People often ask &quot;Why hasn't anybody developed a language with features A, B, C [...]?&quot;</p>
<p>And there's a fairly generic answer that's usually true, and I think it works reasonably well here. It basically comes down to nobody combining the desire and ability to do so.</p>
<p>In this case, on one side, you have people who mostly write UI code. Most of them like languages like C# (to use an example that came up in the comments) and don't like SQL much. They generally use a library/framework to stay as far away from actual SQL as they can. Regardless of how much they might like it if they could actually use it, when it's described that way, they want nothing to do with it.</p>
<p>On the other side, you have DBAs (and such) who care a lot about data consistency, data access, and so on. At least most I've met, have essentially no interest in UIs. Quite a few seem to think an Oracle/Postgres CLI is a perfectly good UI, and couldn't care less about fancy GUI stuff. Quite a few downright sneer at the whole idea of GUIs and nearly anybody who would even consider using one.</p>
<p>One other point of note is that UIs have largely moved to browsers. That leads to most UIs being written in JavaScript or some close derivative such as TypeScript. In theory WebAssembly allows at least some parts to be done in other languages, but in fact it's rarely used--it's more of a science project than a production-ready development environment.</p>
<p>That doesn't leave very many in the middle who like the idea (or even have a reasonable possibility) of doing UI programming (or much of anything else unrelated to data access) with something that resembles SQL. In nearly 15 years on Stack Overflow, and 20 years (or so) on Usenet before that, I've only heard one other person advocate for something even vaguely similar (and at least in the newsgroups I frequented, he was pretty widely seen as one of the worst trolls ever).</p>

CC BY-SA 4.0
2024-12-16T03:33:25.327Z
2024-12-16T06:25:28.337Z
2024-12-16T06:25:28.337Z
Jerry Coffin
CC BY-SA 4.0
2024-12-16T05:53:21.317Z
This doesn't fully ring true with me (although certain parts do!). Notwithstanding different talents and specialisms, the harsh division between DB and UI programmers has arisen in large part because of the complexity and conceptual differences in the tooling available - which if you ask me has got worse in 30 years not better, even though the essential task has often not changed. The first thing the C# programmer will notice, for example, is that strange damned Null concept in SQL that doesn't work like anything he knows in C#. Let's hope I'm not the worst troll ever!
Steve
CC BY-SA 4.0
2024-12-16T06:29:23.290Z
I guess you're free to believe what you wish, but (just for one example) I doubt Nulls would be the first thing most C# programmers noticed about SQL (or even on the first page of things they noticed).
Jerry Coffin
CC BY-SA 4.0
2024-12-16T12:54:21.560Z
That's probably true Jerry, but the real point is that it's the conceptual clash between the two which causes the recoil upon encounter with the other. My question is basically why we don't use aligned concepts throughout all the languages used to build the application - at least to the degree that would not be infeasible for specific reasons.
Steve
CC BY-SA 4.0
2024-12-16T22:50:56.850Z
Oracle's CLI tool manifestly stinks, and the DB2/UDB command tool is even worse.  The Postgresql tool `psql` is everything that a DBA ever needs to maintain a DB, as was `mcr sql$` for Rdb/VMS.
RonJohn
CC BY-SA 4.0
2024-12-17T01:44:34.010Z
@RonJohn: I'm certainly not going to argue that Oracle's CLI tool is good--but I've seen quite a few DBAs who seemed happy with it. Sometimes I think they want it to be miserable, so there's less competition for the job. "I may suck, and the tool definitely does, but my retirement will be glorious!"
Jerry Coffin
CC BY-SA 4.0
2024-12-17T07:06:24.440Z
@JerryCoffin I've often said the same of SQL. It's made me rich.
J. Mini
CC BY-SA 4.0
2024-12-17T15:03:14.927Z
Hmm. I'm not sure this is really picking the right reason. I think it's less that there aren't people who use both SQL and these other languages and more that those people don't think SQLifying the other languages is a good idea.
Jack Aidley
CC BY-SA 4.0
2024-12-17T18:17:10.123Z
"Sometimes I think they want it to be miserable, so there's less competition for the job." That might be part of it, but I think it's even deeper. Using a lot of (especially) older tooling and languages is tedious and error prone. And when you can't really effectively reuse things, it limits acceleration. You come in, make the donuts, and leave.   A steady-state career as long as no one moves their cheese. It might take a while to master it but then you are done and there's very little learning required. I saw this pattern with a lot of COBOL developers who refused to acknowledge its flaws.
JimmyJames
CC BY-SA 4.0
2024-12-18T15:17:45.820Z
Its worth noting that in problem domains that are not interactive, and deal with large amounts of data, relational algebra based libraries are fairly common: Pandas, Apache Spark, Flink, and similar. Even things like samza can be conceptualized in a relational way at an architectural level.
user1937198
CC BY-SA 4.0
2024-12-19T10:59:36.167Z
This answer is implying that backend developers aren't a thing anymore?
Hobbamok
<p>Jerry Coffin already gave a great answer. But let's address some of the requirements you have for this hypothetical language:</p>
<blockquote>
<p>It has various characteristics, including (the ones I think are relevant for this question...):</p>
<ul>
<li>tables as first-class types with associated operators (Join, aggregate, etc.)</li>
</ul>
</blockquote>
<p>What exactly does it mean for <em>a</em> type to be first-class? Maybe you mean built-in type? Otherwise, do you mean something available in the standard library? In the latter case there are many languages have collection types that support operations on them (zip, filter, and so on).</p>
<p>There are not that many languages that have built-in types that are like tables, but I think the <a href="https://en.wikipedia.org/wiki/Array_programming" rel="noreferrer">array programming</a> languages do.</p>
<blockquote>
<ul>
<li>every scalar type has a first-class null value</li>
</ul>
</blockquote>
<p>There are quite a lot of languages that have this built-in (Python is an obvious example, with its <code>None</code>). But often this is seen as a potential drawback, and even in SQL databases you will see <code>NON NULL</code> constraints being used on a lot of columns, with good reason.</p>
<p>There is often some way to make a scalar type nullable in a language, either by using some wrapper (like <a href="https://en.cppreference.com/w/cpp/utility/optional" rel="noreferrer"><code>std::optional</code></a> in C++), or by using a nullable pointer or reference to a value.</p>
<blockquote>
<ul>
<li>a particular (and justifiable) system of how operators handle null values.</li>
</ul>
</blockquote>
<p>They all have a particular system. There are many possible justifications, I think that's not an objective criterion. Of course, crashing when something dereferences a null pointer/reference is probably something you don't want. Even most safe solutions will at least throw some exception when trying to use a null value without explicitly checking for it, but often that is with good reason: just ignoring that something is null is probably a bad thing.</p>

CC BY-SA 4.0
2024-12-16T18:04:15.933Z
2024-12-16T18:04:15.933Z
G. Sliepen
CC BY-SA 4.0
2024-12-16T20:01:38.897Z
Yes, I regard SQL as an array language (albeit one that is peculiar rather than general). I don't think SQL experts regard null as a "drawback" or otherwise questionable though, as it is very intrinsic to how SQL works. Null in SQL doesn't mean a "null reference" - despite the unfortunate homonym, the concept in SQL has quite significant differences that wrong-foot those familiar with other languages. My preferred explanation is that it means the equivalent of "blank" on a paper form - including all the *different* meanings that blank can have.
Steve
CC BY-SA 4.0
2024-12-16T20:41:46.117Z
If you want an example of a language that does have a table as a built-in type, R instantly comes to mind. I think Julia does the same. Generally, any statistics-focused language will have this feature.
J. Mini
CC BY-SA 4.0
2024-12-16T22:54:31.737Z
@Steve "I regard SQL as an array language".  That's so, so, *sooooo* wrong.  And it's why arrays have wriggled their way into SQL.
RonJohn
CC BY-SA 4.0
2024-12-16T23:19:13.437Z
@RonJohn, Codd's relational algebra is essentially about arrays of data, albeit structured in a certain fixed style (the "relation", or table). It's been there from the start. I'd be interested to discuss more if you want to pad your argument out further.
Steve
CC BY-SA 4.0
2024-12-16T23:47:15.897Z
@Steve relational algebra is about *sets* of data.  There's a substantive difference between sets and arrays, if for no other reason than because sets are a concept, and arrays a big chunk of memory.
RonJohn
CC BY-SA 4.0
2024-12-17T00:34:47.650Z
@RonJohn, I'm not sure there's any practical difference in computing terms. A set is just an array with certain mathematical properties attributed to it. You often get a mismatch between how mathematicians and practitioners think about these things. For example, the mathematician may think of sets as unordered. For the practitioner, no data inside a computer is ever without order - there are only algorithms that are non-preserving of an existing order or which cause unprincipled orderings of data, but there is always some order. And unlike RA which uses sets, SQL uses multisets/bags anyway.
Steve
CC BY-SA 4.0
2024-12-17T01:39:36.760Z
@RonJohn: Also, arrays have order. By design, sets of data in SQL do not. You can order the *result* of a SQL query, but at least from a viewpoint of relational theory, if there's any order to records, what you have isn't relational.
Jerry Coffin
CC BY-SA 4.0
2024-12-17T09:56:49.220Z
@Steve randomness is "*some* order".
RonJohn
CC BY-SA 4.0
2024-12-17T14:13:20.283Z
@JerryCoffin, SQL doesn't fully match RA in that regard. Order is important for indexing, and there are corner cases where the order of certain operators can be relied upon. I think the correct mentality to have in SQL is that operators are generally non-conservative of any prior order, not that orderings are absent from data sets.
Steve
CC BY-SA 4.0
2024-12-17T14:45:37.443Z
@RonJohn, the orderings are usually far from random though, and whilst it may seem like I'm dwelling on a very technical point about the difference between randomly ordered and unordered, it's important to recognise firstly that in real SQL engines, order-conservation is a property of the particular operators/algorithms applied to the data, and secondly absence of a principled order is not an intrinsic property of a data set itself. Most data is maintained in durable storage, and emerges from storage, in some kind of specifically-defined indexed order. (1/2)
Steve
CC BY-SA 4.0
2024-12-17T14:48:17.587Z
The ultimate difference between randomly-ordered and unordered data might be merely the recognition of whether an ordering *exists* or not. Those who insist the ordering doesn't exist, although it would seem a tolerable fiction in many cases, will have their basic conceptual notions upended once they encounter a full range of SQL database work - one example being the design of indexes, which is a relatively basic task that even novices might do when using temporary tables, by no means an obscure corner. (2/2)
Steve
CC BY-SA 4.0
2024-12-17T16:05:01.587Z
@Steve: I think part of why Date (or maybe it was Codd--my memory's poor) originally said it was thinking of things like a cluster of servers, each containing the same data, but each (potentially) with entirely different indexing, so consecutive executions of identical queries with unchanged data could still return the results in completely different orders. And yes, with clustered SQL servers that's still entirely possible.
Jerry Coffin
CC BY-SA 4.0
2024-12-17T20:01:50.943Z
@JerryCoffin, it probably was CJ Date. It's certainly true that consecutive executions of the same query can return data in different orders (if not explicitly constrained by an order-by clause), but you and I here are referring to results having alternative or varying orderings, not denying the very existence of an ordering as the mathematicians would when they say "sets are unordered". As I say it's a very subtle point, but it primarily shifts attention onto data lineages and how algorithms conserve or alter the ever-present ordering at each stage of processing.
Steve
<p>FORTRAN and COBOL both, by design, included persistent storage (using the database technology of the day).  When Structured Query Language was introduced, part of the standardization effort produced a standard for language extensions for languages like FORTRAN and COBOL. So that FORTRAN and COBOL could continue to be used the way they were designed, with first-class intrinsic persistent storage.</p>
<p>It's arguable that no system of K&amp;R 'c'  has ever been popular that simply attempts to run with SQL concepts (like listed above) throughout the design, right through to the front-end, thus resolving the notorious &quot;impedance mismatch&quot;. But then K&amp;R 'c' was conceived as 'a small language, only 32 keywords' that didn't even have i/o, let alone persistent storage.</p>
<p>But the idea that such a language had never been 'devised' is historically incorrect.</p>

CC BY-SA 4.0
2024-12-17T05:04:51.153Z
2024-12-17T05:04:51.153Z
david
<p>SQL <em>is</em> used on the client-side. And not just marginally, but at a <em>massive</em> scale.</p>
<p>The <a href="https://sqlite.org/" rel="nofollow noreferrer">SQLite SQL database engine</a> is used in many client products, <a href="https://sqlite.org/famous.html" rel="nofollow noreferrer">among others</a> in:</p>
<ul>
<li>Adobe Lightroom, Adobe Reader, and possibly other products.</li>
<li>The Airbus A350XWB flight software.</li>
<li>Every Android and iOS device contains dozens if not hundreds of SQLite databases.</li>
<li>SQLite is used in macOS and Windows, as well as many other Microsoft and most Apple applications.</li>
<li>Every web browser uses SQLite, often multiple databases.</li>
<li>SQLite is used in car entertainment systems by Bosch, installed in GM, Nissan, and Suzuki automobiles.</li>
<li>Dropbox uses SQLite in its clients.</li>
<li>Intuit uses SQLite in (at least) QuickBooks and TurboTax.</li>
<li>McAfee uses SQLite in its antivirus programs.</li>
<li>RPM stores its state in SQLite.</li>
<li>The Fossil Source Code Management system uses SQLite.</li>
<li>Skype uses SQLite.</li>
<li>Many embedded devices.</li>
<li>Many IoT devices.</li>
</ul>
<p>SQLite is <em>so widely used</em> that it even prevented the standardization of client-side databases for the web! The standardization process requires to demonstrate interoperability between at least two independent implementations, but it turned out that <em>every single major browser</em> used SQLite to implement it – Microsoft Internet Explorer, Mozilla Firefox, Apple Safari, Google Chrome, and Opera.</p>
<p><a href="https://sqlite.org/mostdeployed.html" rel="nofollow noreferrer">SQLite is the most-widely deployed database engine in the world</a>. Not just the most widely-deployed <em>SQL engine</em>, but the most-widely deployed database engine, in general. It is estimated there are over a trillion SQLite databases.</p>
<p>The overwhelming majority of those are on the client side. In phones, tablets, laptops, desktops, game consoles, TVs, set-top boxes, handhelds, cars, IoT devices, other embedded devices. The number of SQLite databases on the client side absolutely <em>dwarfs</em> not just the number of SQLite databases on the server side, but also the number of other SQL databases as well. DB2, Oracle, SQL Server, PostgreSQL, MySQL, and friends are just a rounding error compared to SQLite.</p>

CC BY-SA 4.0
2024-12-15T20:32:02.843Z
2024-12-15T20:32:02.843Z
Jörg W Mittag
CC BY-SA 4.0
2024-12-15T20:38:41.113Z
No, you've got completely the wrong end of the stick with this. See my comment on the question.
Steve
CC BY-SA 4.0
2024-12-16T15:37:02.940Z
Regardless of which end of the stick was grabbed, there's a lot here to chew on.
hardmath
CC BY-SA 4.0
2024-12-18T17:15:23.577Z
A trillion SQLlite DBs? Really? That sounds like a lot of wishful thinking on the part of someone who has something to gain from SQLLite adoption. I take a 20 on my disbelieve roll...
Roger Hill
CC BY-SA 4.0
2024-12-18T20:18:49.250Z
None of the examples you mention are programs written in SQL.
Bergi
CC BY-SA 4.0
2024-12-18T21:54:37.913Z
@Bergi: Nor do I claim that anywhere.
Jörg W Mittag
CC BY-SA 4.0
2024-12-18T21:59:39.800Z
@RogerHill: There are billions of smartphones (estimates vary between 4 and 7+ billion), each of which has many SQLite DBs. Same goes for every smart watch, smart TV, iPad, iPod, Android tablet, Windows or macOS device, many IoT devices, many car entertainment systems, gaming consoles, every mainstream browser. It may not be a full trillion, but definitely dozens to hundreds of billions.
Jörg W Mittag
CC BY-SA 4.0
2024-12-18T22:07:14.063Z
I count ~150 on my MacBook, for example.
Jörg W Mittag
CC BY-SA 4.0
2024-12-18T23:43:15.677Z
@JörgWMittag But that is what the OP asked about: using SQL as a general-purpose language. Not (client) applications containing SQL technology.
Bergi
CC BY-SA 4.0
2024-12-21T17:00:38.737Z
@Bergi At the time this answer was posted, the question asked about the "client-side", and was later edited to say "full-stack" instead.
kaya3
<p>A RDBMS with its SQL language is a tool to store structured data in one particular way. SQL is what it is because it allows very successful representation of the data structures; and also very powerful optimizations by the execution planner component of the RDBMS. It is a quite low-level language as well.</p>
<p>For example, while a RDBMS like Oracle has a dedicted more general programing language (PL/SQL), and while there is <em>some</em> overlap (i.e. you can syntactically just write your SQL right in the PL/SQL program, which is otherwise procedural and vaguely Pascal-like), a PL/SQL program can be viewed as individual, separate SQL statements interspersed by control logic. So not at all like a language you are looking for, presumably.</p>
<p>The power of SQL (in regards on very important aspects like execution planning, optimization, usage of low level hardware resources like buffers, RAM, undo and redo, the ACID principle and so on and forth) lies in the fact that the RDBMS only has to look at a single SQL statement ever, at a time, and that said statement is static, unchanging and arguably extremely simple, once passed to the parser.</p>
<p><em>Any</em> language you put on top that – meaning anything that adds control structures, variables outside of the SQL itself, and all the things that make up a useful language – would break that mold. At that point, you have to see that your language also has a function on top of working with the data: it is the glue between the RDBMS and all other aspects of a functioning software, i.e. the GUI, any kinds of APIs in the modern world, other side effects, and so on and forth. And <em>those</em> things are not something that shout &quot;do me like SQL does&quot;.</p>
<p>Inventing such a general language would be a huge undertaking, and it would be very unclear what the benefit would be. It would very likely end up in the big heap of interesting, fun, or even powerful languages (like many functional or logical languages) that see very little usage in the real world. It would also have to be very tightly coupled to the RDBMS of choice to make any sense, so even if a company like Oracle would decide to invent such a thing, it would be a fringe language used just for that RDBMS.</p>
<p>Finally, the question of how to work with RDBMS from a modern programing language has been solved long ago. These days, all major languages have well-established frameworks that make it as painless as you want it to be. There just is no need.</p>

CC BY-SA 4.0
2024-12-17T11:55:12.267Z
2024-12-17T11:55:12.267Z
AnoE
CC BY-SA 4.0
2024-12-17T14:59:08.583Z
*"that make it as painless as you want it to be" - (roaring laughter).
Steve
CC BY-SA 4.0
2024-12-25T21:14:04.600Z
I'm happy to have sent a smile your way, @Steve. ;)
AnoE
<p>The reason why SQL has not been adapted as a full stack language is because it is frankly, rather awful as a language.</p>
<p>Several things to consider: OP asked about the SQL language, not the underlying RDMS system that it connects to. This is a lot of complex logic to impliment ACID principals on a specific set of hardware.</p>
<p>I think the underlying question was really asking,'I like SQL syntax, why can't I just use it everywhere?'</p>
<p>SQL is written with the assumption that you are largely performing set operations on data to transform it. The 'how' things are done is abstracted away in SQL, but in a lot of cases, this sort of detail is  actually rather important.</p>
<p>Most coding problems are just simpler to work with under a model of imperative commands. SQL has the ability to run in an imperitive fashion, if you want to dive into cursors (but, dont). However, these tend to be very slow and difficult because SQL is optimized for set like operations.</p>
<p>But back to the The SQL language - it isn't very good as a 'programming' lanuage. It is internally inconsistent, and it lacks many basic features essential to work as a universal coding tool. Unless you are using a vendor extended version, it doesn't even have variable support. It is intended to basically be a stateless scripting language. It might be turing complete, but you would have to use the underlying RDMS to make that work. It would need a lot of extension to become a univeraslly useful language, and then it wouldn't probably look much like SQL anymore.</p>
<p>I am guessing that the OP is a new(ish) coder who has not explored other languages yet. Go try some out, there is a richness in diversity. Check out Lisp, that has set like operations and can be used to build actual apps.</p>

CC BY-SA 4.0
2024-12-18T17:38:00.333Z
2024-12-18T17:38:00.333Z
Roger Hill
CC BY-SA 4.0
2024-12-18T17:53:24.867Z
Unfortunately, it wasn't about the syntax! You've also confirmed a thesis I mulled earlier, that part of my problem was that I didn't succeed in telegraphing the depth of my expertise!
Steve
CC BY-SA 4.0
2024-12-19T19:19:39.057Z
If it isn't about the syntax then, I would double down on my suggestion to look at Lisp or some of the other languages that treat data in a set like fashion. That might be the closest thing to what you are looing for.

Or if you are a coding expert, go invent S++, the full stack, turing complete version of SQL.
Roger Hill
CC BY-SA 4.0
2024-12-19T21:50:38.927Z
Thanks Roger, but Lisp is not the sort of language I was thinking about - not because it is not a powerful language, but because there is no connection to SQL in the design. I'm very sceptical that two languages designed independently would be found to marry up well, and there's no substantiation that they do marry. I did a brief search for some kind of connection, and the horror of doing SQL-like things in Lisp can be seen here for example: https://mattpo.pe/posts/sql-but-lisp/. Also I tend to be more sympathetic to procedural languages, especially with ALGOL-style syntaxes. (1/2)
Steve
CC BY-SA 4.0
2024-12-19T21:52:29.373Z
Regards inventing programming languages, I tend to think that useful languages are at least the product of a collective of people whose full-time employment is concerned with design, and whose minds are properly oriented towards the various industrial developer constituencies (rather than the fancies of academics). Think IBM or Bell Labs, or Grace Hopper in the US Navy. I know the remark wasn't intended in the following manner, and the underlying issue here is just a deep disconnect, but in telling me to go invent my own, you may as well be telling me flippantly to "go play in traffic"! (2/2)
Steve
CC BY-SA 4.0
2024-12-20T00:16:38.863Z
If you're claiming to be a renowned civil engineer, it makes _some_ sense to say "go play in traffic" when you ask about a weird new crosswalk design. It doesn't excuse what could be considered attempted murder, but if you're looking for something weirdly specific that no one trying to answer your question is apparently getting at, why _not_ make it yourself?
Someone
CC BY-SA 4.0
2024-12-20T01:24:05.667Z
Well, I have no idea about what your goal is then. If you aren't in it for the syntax, or the set like data operations, I don't see what you are looking for. Putting things in abstracted storage layers?

Remark about inventing it yourself wasn't flippant. Lots of brilliant things were made by engineers who wanted something that nobody else was building. If you want something that doesnt exist, go build it. Good luck!
Roger Hill
CC BY-SA 4.0
2024-12-20T10:43:49.993Z
@Someone, because I thought there would be more people here living in the real world where the question I've asked - about the reconciliation of the world's premiere database language, and other languages that seemingly always have to be used to interact with it - is not "weirdly specific".
Steve
CC BY-SA 4.0
2024-12-20T11:07:04.997Z
@RogerHill, so I've explained to you that it came across flippant, and you've just repeated the same line, without taking stock of why I might have thought that? The vast majority of useful things are made by legions of paid professionals. Have you checked whether I'm leading such a legion, or that I'm even engaged in language design professionally, where it might be feasible for me to follow your encouragement? (1/2)
Steve
CC BY-SA 4.0
2024-12-20T11:08:18.553Z
Secondly, since you admit you've no idea what I'm on about, why would it be sensible for you to encourage me to do something you already suspect you don't even remotely understand? Before I proceed to explain my goal further, do you regard yourself as a fair expert on the SQL language - is it something you know deeply as a theoretician with a special interest in the area (for example, are you familiar with the many differences between RA and SQL?) and/or do you use SQL (or have used) as a professional expert on the language? (2/2)
Steve
CC BY-SA 4.0
2024-12-20T17:17:13.723Z
I feel like one doesn't have to be an expert to give an opinion. The response was basically "Clearly, no one else here is giving you language examples you want, so if you want/need one, why not just make it yourself?"
Someone
CC BY-SA 4.0
2024-12-21T17:06:17.067Z
@Steve The entire premise of this site is that languages don't need to be designed and implemented by unique geniuses. To take umbrage with the suggestion that you design and implement your own language, on a site where the basic assumption is that people here are interested in designing and implementing languages, is like going to a ski resort and then taking umbrage with the suggestion to go skiing.
kaya3
CC BY-SA 4.0
2024-12-21T17:13:02.920Z
Also, it is a bit rich to accuse people of being inflammatory one sentence after you labelled other users on this site "silly people".
kaya3
CC BY-SA 4.0
2024-12-21T18:27:48.807Z
@kaya3, I don't agree. The basic assumption is that people are here to exchange information about language design, not to engage single-handedly (or at all) in the design of a new language, and that is not even the way useful languages are normally designed.
Steve
CC BY-SA 4.0
2024-12-22T16:07:00.093Z
Not all languages have to be ubiquitous or "useful" to everyone. If your specific need is a language that can interface between SQL's jankiness and other languages on the development stack, there's no problem with designing a small interface language. Even if you're the only user, there is still _a_ user, which means the language is, by definition, "useful".
Someone
CC BY-SA 4.0
2024-12-22T23:42:21.537Z
@Someone, it seems that you're proceeding from assumptions very different from my own, as well as continuing to misinterpret the question. Firstly, my need wasn't for a third language to interface between two existing languages, SQL on the server-side and something else on the client-side - it was to purge the usage of languages on the client-side which are conceptually incommensurate with SQL. To have a third language which "translates" but does not purge anything, would only add to the deadweight - I'm not sure whether you're in touch with what I mean by this "weight". (1/2)
Steve
CC BY-SA 4.0
2024-12-22T23:43:43.097Z
Secondly, the "usefulness" has to be weighed against the development and maintenance cost of not just the language itself, but the development environment and the integration into a toolchain, and the reproduction of knowledge about all these things into colleagues' minds. Something that demands more labour than it saves is not "useful". And aside from whether the competencies exist, you don't seem to appreciate the amount of labour this would entail. You don't seem to appreciate the extent to which your suggested actions are extreme and unheard-of. (2/2)
Steve
CC BY-SA 4.0
2024-12-23T00:04:44.680Z
Considering how scuffed I've heard some large companies' _production code_ is, it doesn't seem _completely_ outlandish to have some company develop an internal "special sauce" client language that interfaces with the SQL layer. But it would be quite difficult, yes, which is probably part of the reason why no one does it.
Someone
CC BY-SA 4.0
2024-12-23T00:07:21.423Z
But also, I feel like client-side code shouldn't rely too much on the internal DB's structure—it makes more sense to be modular with one's architecture. If the company were to change the structure (for example, creating a dedicated security layer between the database and the client), it would completely invalidate that "special sauce" that was supposed to interface between the client proper and the database.
Someone
CC BY-SA 4.0
2024-12-23T09:35:06.850Z
@Someone, so we are agreed that the suggestion that a sole developer write their own language was a foolhardy suggestion?
Steve
# Why no "full-stack" SQL-like language?
Posted by: username_0 on 2024-12-15
I've been a user of SQL for a long time now, and it is regarded as extremely well-adapted for its various and widespread uses in data processing. I think you'll hardly find a business that owns a computer, where SQL is not somehow involved in their applications.

It has various characteristics, including (the ones I think are relevant for this question...):

* tables as first-class types with associated operators (Join, aggregate, etc.)
* every scalar type has a first-class null value
* a particular (and justifiable) system of how operators handle null values.

There have been passing fads about "object databases", about innovations which question the necessity of the functionality which SQL engines provide, and much commentary over the years about the "impedance mismatch" between SQL and OO languages.

And I can remember earnest attempts to marry SQL together with other programming languages and GUIs, such as Microsoft Access - which married a database engine called Jet, an SQL language based on ANSI-92, the VBA language (practically equivalent to VB6), and a GUI designer/IDE (which for many simple cases at least, could create and represent a database application with "no code").

Access is still extant, but moribund now. Instead focus has shifted onto more modern languages like C#, and ORM layers like Entity Framework which attempt to concentrate the "impedance mismatch" into a translation layer and resolve it there. Belatedly, the C# language got things like nullable types, too.

For me, amongst the many problems with Access was that VBA had no native nullable types - VBA wasn't designed specifically for this application. What VBA had instead was the generic "Variant" type, which besides storing any type could also store an untyped null value. You could also have arrays of unnamed variants. But there were no built-in table types, no table operators, the built-in null handling was lacklustre and not fully commensurate with SQL. The marriage was workable for the time, but you could very much notice the seams in it.

**Question**

What I'm confused about, and which brings me to my question, is why no system of programming has ever been devised that simply attempts to run with SQL concepts (like listed above) throughout the design, right through to the front-end, thus resolving the notorious "impedance mismatch"?

**EDIT:**

To be clear, I'm not talking about database engines typically used for client-side storage, like SQLite, I'm talking about something that peeforms a role like VBA or C# which typically exist between the storage engine and the UI - but not those languages specifically, because I am arguing that they conceptualise things in ways different from SQL which is what creates the "impedance mismatch".
**Tags:** language-design language-ergonomics nullability programming-paradigms data-types
> **Comments:**
> **username_1**: What kind of "front-end" functionality are you imagining? Would something like [Oracle Forms](https://en.m.wikipedia.org/wiki/Oracle_Forms) or [Oracle Apex](https://apex.oracle.com/en/) qualify?  
> **username_2**: Basically, because SQL is not as expressive as other paradigms. You can resolve the "object-relational impedance mismatch" by adopting other paradigms besides OO without just writing everything in SQL, and if you do stick with OO then the "impedance mismatch" is often an acceptable cost.  
> **username_2**: Of course, the other answer is that a lot of applications *do* use client-side SQL databases. For example, Google Chrome stores your history, cookies etc. in an SQLite database; SQLite is also used by the rpm and Nix package managers on the client-side.  
> **username_0**: @kaya3, I'm not sure if I've been unclear somehow, but you're not following my thinking. I'm not talking about client-side storage using an SQL database engine, and I'm not talking about the SQL language itself being used, but about some unspecified language which, although presumably not SQL, nevertheless aligns sufficiently with SQL concepts that there is no "impedance mismatch". As I say, the absence of a null value within the domain of every type, is something that GP languages seem to lack intrinsically.  
> **username_0**: @IMSoP, I don't know those applications first-hand, but from a brief read I suspect they did originally employ broadly the philosophy I'm talking about. But even they say they are now Java-based, and Java (to my knowledge) is not fully commensurate with SQL.  
> **username_3**: Ruby has only a single type (it is dynamically typed, so from a type-theoretic standpoint, it is "uni-typed"), and it has a `nil` value which is a member of that type. Therefore, it fulfills your criteria of a general purpose language, often used on the client-side, which has a null value within the domain of every type. Is that what you are looking for?  
> **username_3**: You might also be interested in some of the discussions on the Wiki, e.g., [ObjectRelationalImpedanceMismatch](https://wiki.c2.com/?ObjectRelationalImpedanceMismatch) and [ObjectRelationalImpedanceMismatchDoesNotExist](https://wiki.c2.com/?ObjectRelationalImpedanceMismatchDoesNotExist) as well as the many discussions that branch off of those two.  
> **username_0**: @JörgWMittag, but SQL *isn't* "uni-typed". It actually has separate types, but with a null value in each. And it has a scalar/table distinction in its types. And type checking, and operators, etc. What you're describing about Ruby sounds much like VBA's Variant type. So VBA - which I mentioned - already fulfils the criteria as you're interpreting the criteria to be. I'm not sure you're following my thinking about the desirability of conceptual *exact alignment* with SQL in certain respects, including particularly it's type system in an exact way. (Not read the links yet btw).  
> **username_0**: @JörgWMittag, regarding the second link about the non-existence of the impedance mismatch, I think that's primarily argued from the perspective of OO programmers whining about the relational model. I'm actually coming the other way, as a programmer who uses the relational model whining about OO languages!  
> **username_2**: @Steve Well, as I said, there are lots of languages where there is no "object-relational impedance mismatch", as that is only a feature of OO languages. And SQL itself is a non-OO language which is used clientside.  
> **username_0**: @kaya3, as I say, I'm not sure you've followed my thinking. There's a *difference* between for example the type system of SQL and even old non-OO languages like C. I'm not sure why you're sustaining the point that SQL is used client-side, when I've already addressed your confusion in detail above.  
> **username_2**: There is no difference between the type system of SQL and the type system of SQL, though. What about SQL makes it *not* an answer to your question? Yes, clearly, I'm not understanding something about what you want to know, which is why you need to clarify. You are asking *"about some unspecified language which, although presumably not SQL, nevertheless aligns sufficiently with SQL concepts that there is no "impedance mismatch""*. If it's only *presumably* not SQL, then you haven't ruled out SQL as a possible answer. And I don't see anything else which rules out SQL.  
> **username_0**: @kaya3, no but there's a blatantly obvious difference between the context of use, and I've given more than enough detail in my question to allow a sensible person to perceive that aspect. If you think I can replace, say, VBA with SQL - using SQL in places I would usually use VBA to show screens and set UI fields and display message boxes and react to button-presses and so on - then (to say the least) there's a considerable scope for use of SQL which I have not been previously aware.  
> **username_2**: @Steve The only way for you to assert that you have given enough information for a sensible person to understand what you mean, is if you say everyone commenting or answering here is not sensible, because it seems *nobody* other than you understands what you are asking.  
> **username_1**: Nullable types are an extremely common feature of many languages, though implemented in many different ways; C# didn't belatedly add *nullable* types, it added *non-nullable* types (something which SQL also largely lacks). That's completely unrelated to the "object-relational impedance", and any other aspect of SQL.  
> **username_0**: @kaya3, so what role did you see the example of Access and VBA, and my specific reference to the absence of nullable types in VBA despite being married to an SQL engine, playing in my question?  
> **username_0**: @IMSoP, maybe I should have been more specific and said *nullable value types*? At any rate, even the types it has now are not quite full-square with the concept of the null value in SQL.  
> **username_1**: @Steve No, nullable value types have been in C# from the beginning, as have implicitly nullable reference types; what's new is having a context where reference types can error if not *explicitly* nullable. Meanwhile just about every DBMS has a procedural language which builds on SQL, using the same type system as the main database (which may or may not match ANSI SQL). You really do need to be a bit clearer in the question about what you mean by "similar to SQL", and what a language needs to be able to do beyond that.  
> **username_0**: @IMSoP, from what I can see from reading now and vaguely recollect, the last time I used C# on any regular basis (around v5.0), the language was in a condition where nullable types were virtually unusable, because they were explicit containment structures for the base value type with a bolted-on null field, not additional values in the domain of the basic type. Strings were treated differently. And they were all inconsistent with DBNull, as used by many library facilities. I'd practically forgotten the depth of this mess that prevailed at the time! (1/2)  
> **username_0**: So in my view, this is not a point well-made that C# is a language whose type system aligned with SQL "from the beginning". I concede it has shaped up better in intervening years. There's at least a couple of reasons I'm discounting it though. First, C# as a system of programming has not been designed from the outset to align with SQL, and we're arguing here over a small and belated concession/easement, not showing full alignment. And second, the design does not proceed through to the front-end - for example, Winforms has no consistent built-in representation for nullable fields. (2/2)  
> **username_0**: @IMSoP, and just to make my question emphatic, it's not "why can't various languages be pressed reluctantly into interacting with database engines", it is to ask why no language (that I'm aware of) appears to have been designed in which the initial, central, overarching goal was to avoid *any* inconsistency with SQL conceptualisations, right through to the UI layer. I'm not talking about strict syntax alignment with SQL either. (1/2)  
> **username_0**: I'm talking about such things I mentioned in my question like table types, nullable types as they are in SQL, operators that handle nulls the same as SQL, and a reasonably suitable syntax reflecting the central design goal of conceptual alignment. The reason I think it's plausible to have expected such a thing by now, is the ubiquity of SQL engines, and the extent to which a great many bespoke applications fundamentally are client-server database applications. (2/2)  
> **username_4**: For what it's worth, I don't find the intent of this question that hard to discern, although I agree that the title & introduction probably don't get to its point enough (and I don't think the present answer is a fair response to the question). However, there *are* a number of programming languages tightly coupled with databases that seem like they could be counterexamples - are you including/excluding things like MUMPS, Visual FoxPro, many so-called fourth-generation languages? Is the full relational model of ANSI SQL in particular the key aspect? Are non-commercial research systems included?  
> **username_4**: Looked at another way, would the answer "these do exist, they just aren't popular" be a frame challenge to the question, or a direct answer to it?  
> **username_0**: @MichaelHomer, I would say integration with SQL is a key aspect, because SQL is the thing which is widespread. I've heard of MUMPS, but (iirc) it's a medical system of ancient design and not at all SQL-based. FoxPro covers similar ground to MS Access, and (I believe) relates to SQL in a similar (equally deficient) manner. I'd consider it an answer if something did exist, and there was some theory about why it was unpopular. I'd just emphasise/clarify again that I'm not necessarily looking for something which is syntactically identical or even similar to SQL. (1/2)  
> **username_0**: And I'm assuming there would be no significant query functionality in this client-side language. But there would have to be a native representation of tabular data sent from the server, the type system working the same so that every scalar value representable in SQL on the server side has a direct equivalent on the client (so that there is zero need for any translation code, or mental shift by the programmer), etc. (2/2)  
> **username_1**: These comments have drifted a long way from what they're intended for, which is helping to improve the question. So, here's some concrete suggestions for an edit (as in rewording the whole thing, not adding more sections on the end): 1) drop the term "client-side", because you're using it in a non-standard/un-common way; 2) come up with a clear list of criteria that a language would need to match your idea, so we don't have endless No True Scotsman debates; 3) ask about why none have _succeeded_, rather than "devised"  
> **username_0**: @IMSoP, (1) anyone familiar with a client-server database application will understand that term in context of the question, and you're not suggesting any better word; (2) I don't want to exhaustively define the criteria, instead I want you and others to get with the gist of question (or withdraw if your confusion is such that you cannot follow); (3) I'd be happy to know which had been devised but then failed - I've been believing that there hasn't been an attempt. Even allowing for my own muddled terminology, I'm shocked by the level of quarrelsomeness this question is provoking.  
> **username_1**: @Steve Evidently people did *not* interpret "client-side" how you intended. That doesn't make you wrong, or them idiots; but it does mean that you might get more satisfying answers if you can think of a different way to express it. My suggestion was made in good faith: if you want to reduce the confusion and debate on the question, try clarifying it; if you're not interested in clarifying it, don't expect everyone to suddenly understand it.  
> **username_0**: @IMSoP, the point of consulting a community of fellow experts is that they can get with it quickly and provide useful information, it's not supposed to be like confronting a gang of unruly philosophers. I admit "client-side" could be an ambiguous keyword on its own, but I'm less willing to admit that somebody who had read the full question could have attributed to it any meaning other than the one I intended. I'm not sure anyone was actually confused about that point. You commented in a way that suggested you understood my question. (1/2)  
> **username_0**: And Jorg and Kaya responded in ways which suggested they fully got the meaning of "client-side", since they both referred (in responses which were bizarre for other reasons) to the common use of the SQLite storage engine on the client-side. Michael and Jerry have managed without quibble on the point. So what other term to use? If I had mistakenly overlooked a common and more precise term, I could fix. But you have no better suggestion, despite claiming my usage was unconventional, so why raise the complaint? (2/2)  
> **username_5**: I've wondered about this issue for a long time too. (And understood it on the first read). Hope to see interesting answers  
> **username_1**: I'll lay my cards on the table: I have an idea what _might_ be being asked, and have some thoughts on how to answer that interpretation. But since anything I spend effort writing seems likely to receive a reaction of "you're missing the point", or worse that I'll be labelled "bizarre", "idiotic", and so on, I'm not particularly motivated to bother.  
> **username_6**: I'm pretty sure I've seen something fitting the rough description built on XQuery, if you're willing to allow that much deviation from SQL (into general-purpose "query language" space).  
> **username_0**: @IMSoP, I'd certainly be interested to hear your perspective; I'm hoping curiosity causes me no further regrets today. I'm still digesting what all this trouble meant.  
> **username_7**: Computer science professor Ron van der Meyden summarized the various issues as: "The inconsistencies in the SQL standard mean that it is not possible to ascribe any intuitive logical semantics to the treatment of nulls in SQL."  
> **username_0**: @Yakk-AdamNevraumont, I've discussed these interesting points before with other people. Part of the problem is that the null value is not supposed to have one consistent intrinsic meaning, and it is in fact used for multiple different and unrelated purposes in SQL, for very good design reasons. The "treatment" reflects a well-balanced settlement of these different uses. Theoreticians and educators often don't like this complexity, but their objections to it are often naive.  
> **username_7**: @Steve It was used for multiple different and unrelated purposes because people writing SQL where just bashing things together and picked something that sort of made sense. Treating quirky language choices as "very good design reasons" is language-hubris! More importantly, because NULL has no coherent meaning, extending the meaning of NULL to more cases than the eclectic mix of features SQL has has no coherent solution. How should NULL be handled when passed to an operation SQL lacks? You get to make it up, like the designers of SQL did at the time, and you'll get quirks.  
> **username_1**: What "all this trouble" means is that sometimes communication isn't easy, and what seems straightforward and clear to you isn't necessarily as clear to other people, so sometimes you have to put in a bit of effort to choose different words or clear up distracting side points, so that people aren't talking past each other. Repeatedly telling people that they *should* understand, and are idiots if they don't, rarely helps.  
> **username_8**: Your question puts me in mind of [data oriented design](https://en.wikipedia.org/wiki/Data-oriented_design), which often takes the form of "table based" approach to programming. I don't know if there's a mainstream DOD-based programming language yet (akin to Java/Smalltalk for OOP), but I'd imagine someone is researching it.  
> **username_0**: @IMSoP, well one thing I've taken on board is that when I'm asking a question - as a computing expert with considerable general knowledge already - it is almost certain to be something highly obscure, whose implied details would probably be obvious only to others with comparably deep expertise, and I'm implicitly looking for someone who already knows more than me (including whatever I can already gain from research). It might not always be obvious to others how little they know. (1/2)  
> **username_0**: I noticed contemporaneously how quickly I was getting riled in the role of a questioner, compared to my relatively long patience in my usual role as an answerer. I think the root of this - besides my own towering arrogance - is that I'm not particularly well-known here and I obviously hadn't made my grade of expertise sufficiently clear, which perhaps would have produced more pause for thought before some of the less useful replies were plied. (2/2)  
> **username_0**: @R.M., yes it's a good point. What they're calling "data-oriented design" is basically just a return to the status quo ante before OOP, of structured programming, procedural code, and data separate from algorithms.  
> **username_0**: @K.A.Buhr, Haskell? Good heavens, the best cure since they gave electroshock to Hemingway.  
> **username_9**: I think you're looking for Informix-4GL, Natural/ADABAS, 4Js Genero, OpenEdgge ABL, OpenROAD, etc.  
> **username_0**: @user71659, yes I'd say (as a non-user who has quickly looked them up...) those are potentially languages meeting the criteria. Have you any insight to offer as to why they aren't as popular as SQL itself?  
> **username_10**: PowerBuilder is an SQL based language for building thick client front ends off an SQL database. I would not recommend - having to read it in order to port away is bad enough.  
> **username_9**: @Steve I suspect the issue is developers don't want to learn a language and build skills with a narrow application space, coupled with the fact that there's no clear winner and that ends up in a proprietary stack.  
> **username_0**: @user71659, that aligns with my suspicions.  
> **username_11**: @Steve "Have you any insight to offer as to why they aren't as popular as SQL itself? ". Mostly because while _very_ well suited to doing SQL, they're not very well suited to doing anything else. I've worked a lot with Informix 4gl and Genero over the years, and they are _not_ fun languages to work with, despite the many improvements Genero has made over its predecessor.  
> **username_11**: As a general rule, you can't hire developers who already know those languages... and if you ask the developers you _do_ hire to work on them, they tend to resign because they'd rather work on something that's more fun and/or looks better on their CV.  
> **username_12**: Regarding people (mis)understanding your question; your texts are very slow to come to the point, and use a lot of words to say very little. Some concrete examples: _"... it is regarded as extremely well-adapted for its various and widespread uses in data processing."_ - All weasel words and fluff saying nothing. - _"It has various characteristics ..."_ - Of course it does. _Everything_ does. - _"now that's a pugilistic manner of entrance!"_ - Those are your first words in one of your comments here. What even? - You may want to consider improving the clarity of your writing.  
> **username_0**: @marcelm, at times I'm tempted to think I use too many words, but then the extremity of misunderstanding suggests I've used far too few. The purpose of certain sentences is to show what thinking has produced the question (such as about the well-adaptation of SQL, showing clearly that I regard it as the non-negotiable in the overall stack of languages) - I'm happy to sustain an explicit frame challenge, but I don't want people just talking at cross purposes to me, although they have anyway. Hopefully that explains why I regarded the opener of J Mini's answer as fighting talk, said playfully.  
> **username_12**: @Steve I disagree with the premise that more words correlates with more clarity. In fact, I find that often the opposite is true. The key is to ensure every word is important to the essence of your communication.  
> **username_0**: @marcelm, I agree, and I've explained that an example you adduced of fluff actually had a specifically intended purpose when I wrote it. I'm not immune to rambling and wordy sentences, but many of you appear to be just complaining that the question is not sufficiently simple or reducible to a pithy one-liner, when there is no reason it ought to be. As for lack of clarity, I'm not at all satisfied that those lodging complaints have first attempted reasonable interpretation. (1/2)  
> **username_0**: "It has various characteristics..." was used to introduce a list of *relevant* characteristics, and I recall that I first wrote that sentence without the parenthetical remark, then added the parenthesis to emphasise that I was listing an intentional selection. I could reword it to maybe achieve greater artfulness as a sentence, but I struggle to see how I could actually purge words from the overall count, or that it is particularly infelicitous (let alone so inept as to be worthy of your main focus). (2/2)  
> **username_12**: @steve You write, _"I could reword it to maybe achieve greater artfulness as a sentence, but I struggle to see how I could actually purge words from the overall count, or that it is particularly infelicitous (let alone so inept as to be worthy of your main focus)."_ (244 chars) - I might say, _"I could improve it, but I fail to see how to make it shorter, or that it even needs to be."_ (90 chars). That's the same message, in about a third of the characters and much clearer. Your wordiness makes it hard for me to parse out what you actually mean, and I suspect I'm not the only one.  
> **username_0**: @marcelm, nicely done, although it is less specific. I meant the available margin of improvement shows you to be a nit-picker whose attention is misdirected, and I didn't want that to be only implicit.  
> **username_4**: While the responses to this question seem to have gone irretrievably off-track, I do think that there is a useful core question something along the lines of "Why was the approach of the group of general-purpose 'fourth-generation languages' that were tightly coupled with SQL not more widely adopted?". The body would identify the aspects you're most interested in, which I think are particularly creating user interfaces and the relational data model, and the question wouldn't start from the premise that these never existed. This would be a separate, shorter question about design characteristics.  
> **username_0**: @MichaelHomer, I couldn't cope with another question!  
> **username_13**: Those languages exist(ed), they used to be called 4GL languages. Oracle has tools to write full apps with only SQL and PL/SQL (Forms, Reports, APEX). They use mostly the same data types that the database uses. Obviously they're all closed, proprietary and little known tools unless you're already an Oracle customer. And they may look really really reall old-fashioned (Forms and Reports mainly; APEX is usable in modern contexts but still a little hard to version-control).  
> **username_12**: @Steve There seemed to be a lot of confusion as to what you actually meant with your question. All I tried to do is offer more concrete examples of issues I noticed, in the hope to better help you fix this confusion. Do with my feedback what you will, I'm bowing out of this conversation.  
> **username_14**: I'm interpreting this question very differently than the existing answers do. The question I see is: why doesn't the approach SQL was based on get extended to the GUI? I had the same question in the 1990s and I was working on a language to do this, but never finished it. I wanted to explain the research question and my approach to answering it in an answer, but that is impossible when the question is closed. What to do now?  
> **username_0**: @MichaelHomer, is there any chance we can get this question reopened in light of the above commenter's intentions?  
> **username_4**: @reinierpost I've reopened and unprotected the question so that you can post an answer now.  
> **username_14**: @MichaelHomer: ❤️️  
---
## 9 answers
### Answer by username_22 on 2024-12-16. Score: 32
### Because SQL Has Serious Flaws

SQL is an *acceptable* language for doing CRUD on a database. I cannot think of anyone, not even professional DBAs, who would consider SQL a good foundation to build upon. Some major faults come immediately to mind:

* SQL is not an extensible language. I'm not making some Lisp-like argument about how you cannot metaprogram it. My argument is far nastier: most SQLs *don't even have libraries*.
* Unless you're treating SQL like an assembly language and doing performance tuning, [ORMs beat it in every aspect](https://www.linqpad.net/WhyLINQBeatsSQL.aspx). I cannot name *anything* that SQL does better than an ORM.
* Unit testing, one of the cornerstones of modern application development, is extraordinarily difficult in SQL. The language is so limited that a common SQL Server solution is to [use C# to hack it in](https://tsqlt.org/).

None of what I'm saying here is new. You can find lengthy critiques of SQL from [as far back as the 1980s](https://dl.acm.org/doi/pdf/10.1145/984549.984551). Attempts to improve on it, such as [this one](https://www.edgedb.com/blog/we-can-do-better-than-sql), tend not to catch on [because everything is expected to be compatible with SQL](https://www.brentozar.com/archive/2024/06/the-sql-language-is-just-a-hurdle-you-gotta-overcome/).

Your question of why we don't have an application language that solves the impedance mismatch problem or uses `NULL`s just like SQL does (i.e. [inconsistently](https://sqlperformance.com/2020/01/t-sql-queries/null-complexities-part-2)) is entirely sensible, but the question of why we haven't build on SQL to make an application language is much more easily answered by this. SQL just isn't very good at anything other than querying. If you want an application language that handles SQL-like things, then you just use an ORM and you get something *better* than SQL.
> **Comments:**
> **username_0** (2024-12-16): *"Because SQL Has Serious Flaws"* - now that's a pugilistic manner of entrance! I do take the point that not even SQL practitioners regard it as perfect, and that's why I'm not simply asking why we don't generalise the use of SQL, but rather (instead of ORMs) why we don't have languages that maintain certain key conceptualisations that SQL uses. A lot of SQL is ultimately designed for things like efficient algebraic optimisation, concurrent execution with transactional consistency, and online hot-patching, and in these areas it would leave other languages for dust.  
> **username_1** (2024-12-16): One of the things that makes SQL very hard to extend, or generalise, is that its syntax and semantics are both very *ad hoc*. Functions like `EXTRACT` use context-sensitive keywords that don't follow any general grammar, and each major statement has clearly been designed top-down, rather than composed from reusable components. PostgreSQL is among the most flexible implementations, and it's probably no coincidence that it started as a *non-SQL* relational database (Ingres -> Postgres -> PostgreSQL)  
> **username_1** (2024-12-16): @Steve I think the fact that SQL is a) dominant, and b) not very well designed *is* a good reason why no such language has succeeded: if the selling point is "the procedural/functional/imperative/OO part may look weird, but the query part is familiar", you have to start with SQL and botch yet more onto it. The alternative is to have a language where programmers have to learn a new query language *and* a new programming paradigm for the rest of their program. If there was a thriving ecosystem of query languages, there might be more chance that one would evolve into something general-purpose.  
> **username_0** (2024-12-16): @IMSoP, there's definitely lots of non-generality in SQL. What I imagine for this client-side language is not something that reproduces the syntax of SQL (or its oddities) nor really the query functionality. I keep coming back to this touchstone about the representation of null in SQL. For example, in SQL, nullable types are not composites of non-nullable types and an additional flag - they are not sum types. The null value exists directly in the domain of each data type. There are in fact no non-nullable data types - there only constraints that stop nulls being stored in certain cases. (1/2)  
> **username_0** (2024-12-16): These subtleties are actually quite crucial to how SQL works systematically, and they seem often to be misunderstood. I don't want to reduce the whole question to nulls, but I would expect this mystery/new language to at least have *exactly* these data types available in which *no* structural distinction is made between the ordinary values of each data type and the null value which each data type also has. But the syntax can certainly vary from SQL. (2/2)  
> **username_19** (2024-12-16): ORMs better than SQL?  Hah!!  That crap that ORMs generate has to be seen to be believed.  
> **username_22** (2024-12-16): @RonJohn I know that they generate bad SQL. That's why I specifically mentioned performance. _I have seen their evil_.  
> **username_23** (2024-12-17): That edgeDB page also has a good [critique of SQL](https://www.edgedb.com/blog/we-can-do-better-than-sql#critique-of-sql) on it which was very much worth reading.  
> **username_0** (2024-12-17): @JimmyJames, it was worth a read, but I'm not sure I agree with that document at all. They start off with a point about the syntactic similarity of "table and scalar expressions", but in fact the examples are both table expressions. What they identify is the implicit cast of a one-row, one-column table to a scalar - what APL calls "floating". The only heading I was minded to agree with was "lack of consistency". I'm not sure they reveal sufficient familiarity with SQL concepts to be proposing innovations to it.  
> **username_24** (2024-12-18): please explain in what way ORMs are better than SQL.  
> **username_25** (2024-12-18): @Steve: " The null value exists directly in the domain of each data type. There are in fact no non-nullable data types" - why would you want to design a language that bakes in the [billion-dollar mistake](https://en.wikipedia.org/wiki/Tony_Hoare)?  
> **username_0** (2024-12-18): @psmears, because Tony Hoare was talking about the null memory pointer. It's a fairly different concept, and has no analogy in SQL which has always been completely memory safe in that regard. I can't emphasise enough that the homonym does not imply conceptual similarity.  
> **username_25** (2024-12-18): @Steve: The concepts _are_ similar, though; in both cases you're adding a new value to a domain, that needs careful special-case handling. It is not obvious what the right semantics for NULL are - what's correct depends on the situation (ask three programmers how NULLs should behave and you'll get five different answers!). NULL and NULL-like constructs have their uses, but forcing _all_ datatypes to include them invites more harm than good. That's why modern languages don't tend to do that.  
> **username_0** (2024-12-18): @psmears, they really aren't that similar. The perceived similarity is in the general purpose of use, but it's like saying that because we can eat either insects or pigs for food, therefore insects and pigs are the same things. Indeed the analogy you are perceiving is between foreign keys and pointers. A foreign key can be null in SQL, but null is not used exclusively in the context of foreign keys but has substantial other roles too. And unlike the null pointer which is an alias for the zero pointer, the SQL null is not an alias for any other value. (1/2)  
> **username_0** (2024-12-18): Most of the quibbles about null - some expert ones are justified, but many I've heard over time are naive - relate to how the various operators handle nulls, not to the existence of the null values themselves in the domains of all the data types, which is fundamental to SQL and without which it cannot work anything like as we know it. Usually, the issue with null-handling is that SQL takes one approach in all cases when there should be more than one approach for different cases. I can't immediately think of any example where SQL just takes the wrong approach. Try me if you like. (2/2)  
> **username_22** (2024-12-19): @Steve The issue with how SQL handles null is that some parts of the language treat is as "applicable, but unknown" but others treat it as "inapplicable". For example, nullable bits ought to have a four-value logic but SQL has them as a three-value system. My answer links to one article about this.  
> **username_0** (2024-12-19): @J.Mini, that was Codd's later view, that there should be 4VL, but the problem with distinguishing missing and inapplicable, is that the prior question of whether the value is missing or inapplicable can also be unknown. See for example what I wrote on the topic here: https://langdev.stackexchange.com/a/3418/1411. In my view, this approach to improvement is misguided. The actual role of null is as a kind of ground state, a dustbin for *all* the reasons why the data cannot take on a more specific value. (1/2)  
> **username_0** (2024-12-19): The 4VL approach seemingly attempts to make null fully-specific in its meaning again, denying the dustbin/groundstate role, the very existence of such a role being what contributes the majority of its usefulness (unless you go to 5VL, and have missing, inapplicable, and groundstate). My experience is that missing/inapplicable don't tend to need distinct *handling* (I can recall occasional cases where the distinction was useful to propagate to the user, but not where the automatic handling differed). (2/2)  
> **username_0** (2024-12-19): @J.Mini, just picking up another track here, you link to a unit testing framework for SQL. The problem with unit tests is that my experience of working even with very weak SQL developers is that their main faults manifest in three areas: (1) fundamental analysis faults where they haven't grasped what a correct result looks like even in principle; (2) concurrency faults/performance emergencies which only system/integration tests could detect; (3) carelessness in general. Although not zero, the number of faults where unit testing seems likely to catch, are a small minority.  
> **username_0** (2024-12-19): @Hobbamok, the problem with ORMs is that the vast majority of applications aren't confined just to the simple cases they handle well. If you have a legion of low-grade application programmers plus a few expert SQL developers to take over all the hard cases or performance hotspots, then it might at least facilitate a division of labour, but for any developers who have to handle the whole stack (including any maintenance developers/DBAs later) it introduces an additional layer of complications.  
> **username_0** (2024-12-19): @Hobbamok, I'm just saying most multi-user applications simply *don't* fit in that simple case. It doesn't invalidate your point, it's just emphatic about those "shitshow" cases, and about the division of labour aspect (it's more sensible when there's a division of labour and releases some developers from needing any SQL skills, than for full-stack developers who still need all the skills).  
> **username_0** (2024-12-19): @Hobbamok, when I said "multi-user", I didn't have a specific threshold in mind but it was more like, you know, 20 or 50 concurrent sessions - far from multi-millions!  
> **username_26** (2024-12-20): The citation for ORMs beating SQL is about how nice their syntax is. This question, on the other hand, has nothing to do with syntax but asks generally about languages with SQL-like semantics, a category that includes LINQ. LINQ has the same thing as SQL, that it only applies to data queries and cannot be used to write a whole application front-to-back.  
> **username_0** (2024-12-26): @Hobbamok, I find that quite remarkable really, that a company can commission a database application for 20 or 50 *concurrent* users, and not have anyone involved in the development with real database skills and experience (probably explains that EAV disaster I once saw!). I did concede ORMs have a valid point - to enable a division of labour between those who can with SQL, and those who cannot.  
> **username_0** (2024-12-26): @MisterMiyagi, I concede your point, but the primary use-case for client-server SQL engines, is for many users to be concurrently reading and altering the same data (which is usually has significant volume and variety). Concurrency and volume, those are why you have to use SQL engines and not any other. If the application has neither concurrency (in the relevant sense) nor volume, and if you have nobody on the team who is a competent SQL developer, then it more raises the question why it's perceived as desirable to use an SQL engine without a single SQL *engineer*? Then an ORM layer on top?  
> **username_0** (2024-12-26): @MisterMiyagi, I agree and I know this, but question is, why would you use SQLite, if all you have on your team is people who *don't* know any SQL? For a single-user business application, with DotNet for example, you could just serialise the entire in-memory object graph - probably without writing more than a few lines to set up a built-in serialiser. I just don't understand how you get from "we're a team that doesn't know any SQL now, and won't learn!" to "we'll use SQLite plus an ORM framework for local storage in our simple single-user application!"   
> **username_0** (2024-12-26): @MisterMiyagi, *I'm* not arguing against SQLite. You don't need to explain these advantages to me - I'm a fair expert SQL and pro-SQL. You need to explain why a team which knows little or nothing about SQL, can and do nevertheless appreciate all these advantages of using the relational model and an SQL engine? I find it relatively implausible that a team who have no SQL skills themselves and only simple (single-user, low-volume) storage needs, would nevertheless adopt these supremely positive attitudes towards the use of SQL engines and ORMs? Why do they need indexing if it's low-volume?  
> **username_0** (2024-12-26): @MisterMiyagi, it's one thing to know the SQL syntax and dislike it. I can follow the logic about how a person who knows SQL and understands relational databases, but vehemently dislikes SQL, can learn an ORM and use that - and even if I'm saying this is less efficient, his hatred will drive him through, and he can still revert to the SQL he knows and hates whenever the ORM reaches its limits. But I'm still not convinced by your last assertion - you're not talking about people who know and hate SQL, you're talking about people who don't even know how SQL works in the first place.  
> **username_0** (2024-12-26): @MisterMiyagi, I understand the distinction you're making, but in practice I still find it very doubtful that someone would have a good working grasp of SQL engines without knowing any SQL! I struggle to imagine the motivation or the professional circumstances of someone who acquires some kind of working conceptual knowledge of SQL engines, and accepts they are a good useful technology, yet manages to dodge any encounter with the SQL lingua franca of these engines! I'm not even sure why you're trying to establish the existence of this person - do you actually know a real person like this?  
> **username_27** (2024-12-26): I think the point is that SQL engines are not the only kind of database handler out there.  
> **username_0** (2024-12-27): @Someone, why would we be discussing ORMs if the point was about non-SQL databases?  
> **username_0** (2024-12-27): @MisterMiyagi, I began by talking about *normal* database applications. I had two points: (a) that ORMs don't enable strong SQL skills to be eliminated from the development team when writing normal database applications; and (b) ORMs themselves inflict additional overheads (they're primarily designed to facilitate division of labour and skills specialisation, not to boost the productivity of a full-stack developer, the latter of whom will pay the overhead but will not benefit from any division or specialisation).  
---
### Answer by username_28 on 2024-12-16. Score: 15
# Disclaimer

The question asked is specifically *why* this hasn't happened. Not about whether it's a good idea, helpful, etc. So quite a bit of what I have to say is related solely to why things are the way they are, with no comment about other questions that could be seen as related.

# Answer

People often ask "Why hasn't anybody developed a language with features A, B, C [...]?"

And there's a fairly generic answer that's usually true, and I think it works reasonably well here. It basically comes down to nobody combining the desire and ability to do so.

In this case, on one side, you have people who mostly write UI code. Most of them like languages like C# (to use an example that came up in the comments) and don't like SQL much. They generally use a library/framework to stay as far away from actual SQL as they can. Regardless of how much they might like it if they could actually use it, when it's described that way, they want nothing to do with it.

On the other side, you have DBAs (and such) who care a lot about data consistency, data access, and so on. At least most I've met, have essentially no interest in UIs. Quite a few seem to think an Oracle/Postgres CLI is a perfectly good UI, and couldn't care less about fancy GUI stuff. Quite a few downright sneer at the whole idea of GUIs and nearly anybody who would even consider using one.

One other point of note is that UIs have largely moved to browsers. That leads to most UIs being written in JavaScript or some close derivative such as TypeScript. In theory WebAssembly allows at least some parts to be done in other languages, but in fact it's rarely used--it's more of a science project than a production-ready development environment.

That doesn't leave very many in the middle who like the idea (or even have a reasonable possibility) of doing UI programming (or much of anything else unrelated to data access) with something that resembles SQL. In nearly 15 years on Stack Overflow, and 20 years (or so) on Usenet before that, I've only heard one other person advocate for something even vaguely similar (and at least in the newsgroups I frequented, he was pretty widely seen as one of the worst trolls ever).
> **Comments:**
> **username_0** (2024-12-16): This doesn't fully ring true with me (although certain parts do!). Notwithstanding different talents and specialisms, the harsh division between DB and UI programmers has arisen in large part because of the complexity and conceptual differences in the tooling available - which if you ask me has got worse in 30 years not better, even though the essential task has often not changed. The first thing the C# programmer will notice, for example, is that strange damned Null concept in SQL that doesn't work like anything he knows in C#. Let's hope I'm not the worst troll ever!  
> **username_28** (2024-12-16): I guess you're free to believe what you wish, but (just for one example) I doubt Nulls would be the first thing most C# programmers noticed about SQL (or even on the first page of things they noticed).  
> **username_0** (2024-12-16): That's probably true Jerry, but the real point is that it's the conceptual clash between the two which causes the recoil upon encounter with the other. My question is basically why we don't use aligned concepts throughout all the languages used to build the application - at least to the degree that would not be infeasible for specific reasons.  
> **username_19** (2024-12-16): Oracle's CLI tool manifestly stinks, and the DB2/UDB command tool is even worse.  The Postgresql tool `psql` is everything that a DBA ever needs to maintain a DB, as was `mcr sql$` for Rdb/VMS.  
> **username_28** (2024-12-17): @RonJohn: I'm certainly not going to argue that Oracle's CLI tool is good--but I've seen quite a few DBAs who seemed happy with it. Sometimes I think they want it to be miserable, so there's less competition for the job. "I may suck, and the tool definitely does, but my retirement will be glorious!"  
> **username_22** (2024-12-17): @JerryCoffin I've often said the same of SQL. It's made me rich.  
> **username_29** (2024-12-17): Hmm. I'm not sure this is really picking the right reason. I think it's less that there aren't people who use both SQL and these other languages and more that those people don't think SQLifying the other languages is a good idea.  
> **username_23** (2024-12-17): "Sometimes I think they want it to be miserable, so there's less competition for the job." That might be part of it, but I think it's even deeper. Using a lot of (especially) older tooling and languages is tedious and error prone. And when you can't really effectively reuse things, it limits acceleration. You come in, make the donuts, and leave.   A steady-state career as long as no one moves their cheese. It might take a while to master it but then you are done and there's very little learning required. I saw this pattern with a lot of COBOL developers who refused to acknowledge its flaws.  
> **username_21** (2024-12-18): Its worth noting that in problem domains that are not interactive, and deal with large amounts of data, relational algebra based libraries are fairly common: Pandas, Apache Spark, Flink, and similar. Even things like samza can be conceptualized in a relational way at an architectural level.  
> **username_30** (2024-12-19): This answer is implying that backend developers aren't a thing anymore?  
---
### Answer by username_15 on 2024-12-16. Score: 13
LINQ, and null coalescing operator ??

What I'm confused about, and which brings me to my question, is why no system of programming has ever been devised that simply attempts to run with SQL concepts (like listed above) throughout the design, right through to the front-end, thus resolving the notorious "impedance mismatch"?

You may be interested in Language Integrated Query. Complemented with the null coalescing operator ??, it appears to tick all the boxes.


tables as first-class types with associated operators (Join, aggregate, etc.)


Tables, or to be precise, any IEnumerable entity, can be selected, matched, filtered, and have functions applied over, inclusive using near identical SQL syntax. The example below is valid C# code!
var artists = from a in context.Artists
              where a.Name.StartsWith("A")
              orderby a.Name
              select a



every scalar type has a first-class null value


Everything in C# is now nullable.


a particular (and justifiable) system of how operators handle null values.


Testing for == null and skipping nulls with ?? are both possible, both in class and syntax forms.
> **Comments:**
> **username_16** (2024-12-17): ^^^ This. I'm surprised no one else has mentioned LINQ yet.  
> **username_16** (2024-12-17): Although based on the comments to the question, I imagine this doesn't fit the narrow criteria of the OP.  
> **username_17** (2024-12-18): I think this answer supports the answer which suggests the problem is that SQL has serious flaws and the comment about it being very ad-hoc. I found LINQ a bit difficult to wrap my head around because it simultaneously seems to turn SQL inside out (e.g. FROM comes first, SELECT comes last) while also doing little to fix the ad-hocness.  
> **username_0** (2024-12-18): @Michael, putting the from-clause first is actually one thing to which I've never heard an objection! When I write SQL, I often start with a boilerplate `SELECT * FROM` and then proceed to write the actual query, going back to modify the select-clause once the relevant fields are in scope. The confusing thing about Linq - I used to find - was predominantly related to the more complicated data shapes (and their typing) it was capable of working with, inspecting intermediate results, or reasoning about the moment and effects of evaluation.  
> **username_17** (2024-12-19): @Steve Oh I agree, when you think about it, it does make sense that FROM comes first, in fact the order in which SELECT specifies things is a bit out of order from how the query actually works.  I'm not sure if Linq entirely fixes that, but it does take a bit of unlearning.  
---
### Answer by username_18 on 2024-12-16. Score: 11
You ask,

> why no system of programming has ever been devised that simply attempts to run with SQL concepts [...] throughout the design, right through to the front-end, thus resolving the notorious "impedance mismatch"?

Fundamentally, I'd attribute that to SQL having only one form of data organization: the table. That serves well enough (though by no means flawlessly) for data storage, but tabular organization is simply not natural for a lot of the data that applications work with. This is precisely the nature of the "impedance mismatch" in the first place.

That doesn't mean that table data types and relational algebra operators couldn't be provided (and I've occasionally seen them in libraries), but as soon as you give a language any other kind of aggregate data type, you introduce the mismatch that you ask about, between tabular data types and other aggregates. You just get it directly in the application language instead of between the application language and the data access language.

By and large, people making a language choice choose a language that they expect to be one or more of convenient, comfortable, effective, productive, performant, or cost-efficient. Maybe I've missed one or two. A table-oriented language isn't any of that for much of any problem other than storage. Instead, it ticks pretty much every box to have more flexibility. That doesn't mean that such a language couldn't get the job done, just that no one has any motivation to create the language.

SQL is as successful as it is largely because it doesn't have to provide an end-to-end solution. Its success in its niche should not be mistaken as a sign that it, or something like it, would be well adapted to a broader role.
> **Comments:**
> **username_19** (2024-12-16): To summarize: SQL is a domain-specific language.  
> **username_0** (2024-12-16): The tabular model in SQL is actually key to it's flexibility and queryability (so to speak), not restrictiveness. There's no *single* other data structure that is quite as flexible for automatic processing, including ease of mechanical transformation into more complicated shapes (as is often done for final presentation). I'm not saying this mystery new language would have to confine itself only to tables of course, I'm just remarking on the question of whether tables are limiting or not.  
> **username_0** (2024-12-16): @RonJohn, yes but the "domain" here actually covers the majority of the purpose for which organisations use computers, which is automatic data processing. Whilst I understand the concept of a "general purpose" programming language as typically meaning you can write an operating system as much as a business application, the majority of languages and frameworks in use today are actually "domain specific" by that old criteria. So I wouldn't be tempted to think that SQL is doing something narrow - it's a *highly general* language for a domain of considerable size and importance.  
> **username_18** (2024-12-17): @Steve, I think I saw some of the "everything's an object" guys looking for you.  But seriously, the tabular model is flexible in the sense of being able to model data, but not more so than Java or C++ classes, for example, since classes can implement tables.  The tabular model is, I agree, central to querying, but querying is not the be all and end all of application programming.  Data representation matters, programming paradigm matters, and a few other things that tables are not flexible about sometimes matter too.  
> **username_20** (2024-12-17): @Steve You seem to be ignoring efficiency here. You _could_ implement almost any data structure you can think of in SQL using tables and queries. But for almost anything other than basic mapping types and and classes that only store data, it’s horribly inefficient. There’s a very good reason that things like time-series and graph databases exist, the data structures they represent can’t be efficiently implemented using SQL, and _efficiency is important_.  
> **username_0** (2024-12-17): @JohnBollinger, I agree with your points. I'm not thinking of a language that restricts itself to only a tabular model. But since SQL does work exclusively on a tabular model, the mystery language should have a particularly efficient manner of handling tables (which will often be going to or coming from an SQL engine).  
> **username_0** (2024-12-17): @AustinHemmelgarn, yes, I wouldn't want to argue that SQL engines must be best for every conceivable data processing application, although I'm not sure to what extent the problems you identify are due to the tabular conceptual model in SQL (as opposed to the particular design of the storage subsystems of mainstream SQL engines). Rather, I'm proceeding from the premise that their popularity and ubiquity in practice - which has grown despite years of considerable opposition amongst the punditry, whilst the proposed alternatives all flashed in the pan - is proof of the general usefulness.  
> **username_20** (2024-12-17): @Steve Proposed alternatives all flashed in the pan? You must not have heard of the numerous TSDB, document database, graph database, and KV store projects that are doing very well and seeing plenty of widespread usage. Just off the top of my head, MongoDB, Riak, Neo4j and RocksDB are at least as big as the major SQL platforms (other than SQLite, but that is ubiquitous for reasons other than it’s usage of SQL), and there are plenty of others out there as well.  
> **username_20** (2024-12-17): @Steve And as far as the difference between the SQL engine and the data model itself, the issue is more in the data model and the storage formats. Graph structures _cannot_ be handled efficiently within the standard SQL data model due to scaling issues (the data model itself inherently turns things into _n²_ or worse efficiency in either time or space based on the number of nodes or edges involved), and efficient TSDB usage requires different storage than what SQL systems use (and optimizing for TSDB usage there will make ‘regular’ SQL usage less efficient, because they need different things).  
> **username_18** (2024-12-17): @Steve, you write "*I'm not thinking of a language that restricts itself to only a tabular model*", but this answer already speaks to that.  The kind of language you are now talking about does not resolve the impedance problem, but rather internalizes it. Tables are good for storage.  Other kinds of data structures tend to be preferred for processing. The mismatch hits when you want to move between those.  
> **username_0** (2024-12-17): @JohnBollinger, I think the "impedance mismatch" is caused not just by the change in data structure per se, but by the fact that it has to be done right at the interface between two incommensurate languages, rather than inside one language that is designed to neatly handle SQL result sets on their own terms (as well as designed to handle any further transformation that may be desired).  
> **username_18** (2024-12-17): @Steve, this answer already claims otherwise.  And your assertion leads me to think that you may have a slightly incorrect idea about the nature of the "impedance mismatch".  The mismatch is that whereas tabular representation is well suited for storage, it is not especially well suited for many kinds of processing operations, while those representations that are well suited for processing are not so well suited for storage.  Moving the point where you change between those does not reduce that.  Neither does making such changes easier, if indeed you can.  
> **username_0** (2024-12-17): @JohnBollinger, although I'm not being exhaustive about all its aspects, the impedance mismatch is about a clash of conceptualisations, in my view. The tabular model partly emerged because more complicated structures (which had already been tried before the SQL era), whilst they may be highly effective for certain queries, are highly ineffective for others - that is, no single more-complicated form copes better in general than tables. More complicated forms are also more difficult to express queries for (that is, specify the desired processing upon the data). (1/2)  
> **username_0** (2024-12-17): The reason I'm suggesting "moving the translation point" fully into the language, is because by definition the language must then incorporate both conceptualisations and reconcile them somehow in order to facilitate translation of the data between each. Whereas when the translation is done immediately at the point where the data first enters the scope of the client-side language, then you have an unholy situation where the translation awkwardness for the language user is mandatory rather than optional, and where the language designer tends not to take responsibility for that awkwardness. (2/2)  
> **username_18** (2024-12-17): At this point, @Steve, it's clear that we have moved off the question posed, which supposes that one could *resolve* the mismatch by use of a general-purpose language that was substantially SQL-like.  Maybe one could, but, again, no one wants to do application programming in that language.  Now you're suggesting a language that *internalizes* the mismatch and moderates its effects, which is different.  I think if it were as easy as that then it wouldn't need to have a language built around it in the first place, and it probably would have been done already. [...]  
> **username_18** (2024-12-17): [...] But again, that's a different question, not the one to which this answer is directed.  
> **username_0** (2024-12-17): @JohnBollinger, I suppose I'm playing two hands here. I think a client-side language that just runs with certain SQL concepts, would work better for many applications. But I'm also provisionally accepting your point that maybe something about existing client-side languages works better in other cases, and therefore a translation would still be necessary, rather than simply aligning fully to SQL concepts. Anyway I'll leave it there, I suppose, rather than drag you any further into the complications.  
> **username_21** (2024-12-18): One problem tables (multisets) are well suited for other than storage is distributed compute. Apache Spark RDDs are basically another form of relational algebra, but with an object encoding built on top. The non-ordered set properties make distributing the work by partitioning the tuples really simple.  
---
### Answer by username_31 on 2024-12-16. Score: 5
Jerry Coffin already gave a great answer. But let's address some of the requirements you have for this hypothetical language:

> It has various characteristics, including (the ones I think are relevant for this question...):
>
> * tables as first-class types with associated operators (Join, aggregate, etc.)

What exactly does it mean for *a* type to be first-class? Maybe you mean built-in type? Otherwise, do you mean something available in the standard library? In the latter case there are many languages have collection types that support operations on them (zip, filter, and so on).

There are not that many languages that have built-in types that are like tables, but I think the [array programming](https://en.wikipedia.org/wiki/Array_programming) languages do.

> * every scalar type has a first-class null value

There are quite a lot of languages that have this built-in (Python is an obvious example, with its `None`). But often this is seen as a potential drawback, and even in SQL databases you will see `NON NULL` constraints being used on a lot of columns, with good reason.

There is often some way to make a scalar type nullable in a language, either by using some wrapper (like [`std::optional`](https://en.cppreference.com/w/cpp/utility/optional) in C++), or by using a nullable pointer or reference to a value.

> * a particular (and justifiable) system of how operators handle null values.

They all have a particular system. There are many possible justifications, I think that's not an objective criterion. Of course, crashing when something dereferences a null pointer/reference is probably something you don't want. Even most safe solutions will at least throw some exception when trying to use a null value without explicitly checking for it, but often that is with good reason: just ignoring that something is null is probably a bad thing.
> **Comments:**
> **username_0** (2024-12-16): Yes, I regard SQL as an array language (albeit one that is peculiar rather than general). I don't think SQL experts regard null as a "drawback" or otherwise questionable though, as it is very intrinsic to how SQL works. Null in SQL doesn't mean a "null reference" - despite the unfortunate homonym, the concept in SQL has quite significant differences that wrong-foot those familiar with other languages. My preferred explanation is that it means the equivalent of "blank" on a paper form - including all the *different* meanings that blank can have.  
> **username_22** (2024-12-16): If you want an example of a language that does have a table as a built-in type, R instantly comes to mind. I think Julia does the same. Generally, any statistics-focused language will have this feature.  
> **username_19** (2024-12-16): @Steve "I regard SQL as an array language".  That's so, so, *sooooo* wrong.  And it's why arrays have wriggled their way into SQL.  
> **username_0** (2024-12-16): @RonJohn, Codd's relational algebra is essentially about arrays of data, albeit structured in a certain fixed style (the "relation", or table). It's been there from the start. I'd be interested to discuss more if you want to pad your argument out further.  
> **username_19** (2024-12-16): @Steve relational algebra is about *sets* of data.  There's a substantive difference between sets and arrays, if for no other reason than because sets are a concept, and arrays a big chunk of memory.  
> **username_0** (2024-12-17): @RonJohn, I'm not sure there's any practical difference in computing terms. A set is just an array with certain mathematical properties attributed to it. You often get a mismatch between how mathematicians and practitioners think about these things. For example, the mathematician may think of sets as unordered. For the practitioner, no data inside a computer is ever without order - there are only algorithms that are non-preserving of an existing order or which cause unprincipled orderings of data, but there is always some order. And unlike RA which uses sets, SQL uses multisets/bags anyway.  
> **username_28** (2024-12-17): @RonJohn: Also, arrays have order. By design, sets of data in SQL do not. You can order the *result* of a SQL query, but at least from a viewpoint of relational theory, if there's any order to records, what you have isn't relational.  
> **username_19** (2024-12-17): @Steve randomness is "*some* order".  
> **username_0** (2024-12-17): @JerryCoffin, SQL doesn't fully match RA in that regard. Order is important for indexing, and there are corner cases where the order of certain operators can be relied upon. I think the correct mentality to have in SQL is that operators are generally non-conservative of any prior order, not that orderings are absent from data sets.  
> **username_0** (2024-12-17): @RonJohn, the orderings are usually far from random though, and whilst it may seem like I'm dwelling on a very technical point about the difference between randomly ordered and unordered, it's important to recognise firstly that in real SQL engines, order-conservation is a property of the particular operators/algorithms applied to the data, and secondly absence of a principled order is not an intrinsic property of a data set itself. Most data is maintained in durable storage, and emerges from storage, in some kind of specifically-defined indexed order. (1/2)  
> **username_0** (2024-12-17): The ultimate difference between randomly-ordered and unordered data might be merely the recognition of whether an ordering *exists* or not. Those who insist the ordering doesn't exist, although it would seem a tolerable fiction in many cases, will have their basic conceptual notions upended once they encounter a full range of SQL database work - one example being the design of indexes, which is a relatively basic task that even novices might do when using temporary tables, by no means an obscure corner. (2/2)  
> **username_28** (2024-12-17): @Steve: I think part of why Date (or maybe it was Codd--my memory's poor) originally said it was thinking of things like a cluster of servers, each containing the same data, but each (potentially) with entirely different indexing, so consecutive executions of identical queries with unchanged data could still return the results in completely different orders. And yes, with clustered SQL servers that's still entirely possible.  
> **username_0** (2024-12-17): @JerryCoffin, it probably was CJ Date. It's certainly true that consecutive executions of the same query can return data in different orders (if not explicitly constrained by an order-by clause), but you and I here are referring to results having alternative or varying orderings, not denying the very existence of an ordering as the mathematicians would when they say "sets are unordered". As I say it's a very subtle point, but it primarily shifts attention onto data lineages and how algorithms conserve or alter the ever-present ordering at each stage of processing.  
---
### Answer by username_32 on 2024-12-17. Score: 4
FORTRAN and COBOL both, by design, included persistent storage (using the database technology of the day). When Structured Query Language was introduced, part of the standardization effort produced a standard for language extensions for languages like FORTRAN and COBOL. So that FORTRAN and COBOL could continue to be used the way they were designed, with first-class intrinsic persistent storage.

It's arguable that no system of K&R 'c' has ever been popular that simply attempts to run with SQL concepts (like listed above) throughout the design, right through to the front-end, thus resolving the notorious "impedance mismatch". But then K&R 'c' was conceived as 'a small language, only 32 keywords' that didn't even have i/o, let alone persistent storage.

But the idea that such a language had never been 'devised' is historically incorrect.
---
### Answer by username_3 on 2024-12-15. Score: 2
SQL *is* used on the client-side. And not just marginally, but at a *massive* scale.

The [SQLite SQL database engine](https://sqlite.org/) is used in many client products, [among others](https://sqlite.org/famous.html) in:

* Adobe Lightroom, Adobe Reader, and possibly other products.
* The Airbus A350XWB flight software.
* Every Android and iOS device contains dozens if not hundreds of SQLite databases.
* SQLite is used in macOS and Windows, as well as many other Microsoft and most Apple applications.
* Every web browser uses SQLite, often multiple databases.
* SQLite is used in car entertainment systems by Bosch, installed in GM, Nissan, and Suzuki automobiles.
* Dropbox uses SQLite in its clients.
* Intuit uses SQLite in (at least) QuickBooks and TurboTax.
* McAfee uses SQLite in its antivirus programs.
* RPM stores its state in SQLite.
* The Fossil Source Code Management system uses SQLite.
* Skype uses SQLite.
* Many embedded devices.
* Many IoT devices.

SQLite is *so widely used* that it even prevented the standardization of client-side databases for the web! The standardization process requires to demonstrate interoperability between at least two independent implementations, but it turned out that *every single major browser* used SQLite to implement it – Microsoft Internet Explorer, Mozilla Firefox, Apple Safari, Google Chrome, and Opera.

[SQLite is the most-widely deployed database engine in the world](https://sqlite.org/mostdeployed.html). Not just the most widely-deployed *SQL engine*, but the most-widely deployed database engine, in general. It is estimated there are over a trillion SQLite databases.

The overwhelming majority of those are on the client side. In phones, tablets, laptops, desktops, game consoles, TVs, set-top boxes, handhelds, cars, IoT devices, other embedded devices. The number of SQLite databases on the client side absolutely *dwarfs* not just the number of SQLite databases on the server side, but also the number of other SQL databases as well. DB2, Oracle, SQL Server, PostgreSQL, MySQL, and friends are just a rounding error compared to SQLite.
> **Comments:**
> **username_0** (2024-12-15): No, you've got completely the wrong end of the stick with this. See my comment on the question.  
> **username_33** (2024-12-16): Regardless of which end of the stick was grabbed, there's a lot here to chew on.  
> **username_34** (2024-12-18): A trillion SQLlite DBs? Really? That sounds like a lot of wishful thinking on the part of someone who has something to gain from SQLLite adoption. I take a 20 on my disbelieve roll...  
> **username_35** (2024-12-18): None of the examples you mention are programs written in SQL.  
> **username_3** (2024-12-18): @Bergi: Nor do I claim that anywhere.  
> **username_3** (2024-12-18): @RogerHill: There are billions of smartphones (estimates vary between 4 and 7+ billion), each of which has many SQLite DBs. Same goes for every smart watch, smart TV, iPad, iPod, Android tablet, Windows or macOS device, many IoT devices, many car entertainment systems, gaming consoles, every mainstream browser. It may not be a full trillion, but definitely dozens to hundreds of billions.  
> **username_3** (2024-12-18): I count ~150 on my MacBook, for example.  
> **username_35** (2024-12-18): @JörgWMittag But that is what the OP asked about: using SQL as a general-purpose language. Not (client) applications containing SQL technology.  
> **username_2** (2024-12-21): @Bergi At the time this answer was posted, the question asked about the "client-side", and was later edited to say "full-stack" instead.  
---
### Answer by username_36 on 2024-12-17. Score: 2
A RDBMS with its SQL language is a tool to store structured data in one particular way. SQL is what it is because it allows very successful representation of the data structures; and also very powerful optimizations by the execution planner component of the RDBMS. It is a quite low-level language as well.

For example, while a RDBMS like Oracle has a dedicted more general programing language (PL/SQL), and while there is *some* overlap (i.e. you can syntactically just write your SQL right in the PL/SQL program, which is otherwise procedural and vaguely Pascal-like), a PL/SQL program can be viewed as individual, separate SQL statements interspersed by control logic. So not at all like a language you are looking for, presumably.

The power of SQL (in regards on very important aspects like execution planning, optimization, usage of low level hardware resources like buffers, RAM, undo and redo, the ACID principle and so on and forth) lies in the fact that the RDBMS only has to look at a single SQL statement ever, at a time, and that said statement is static, unchanging and arguably extremely simple, once passed to the parser.

*Any* language you put on top that – meaning anything that adds control structures, variables outside of the SQL itself, and all the things that make up a useful language – would break that mold. At that point, you have to see that your language also has a function on top of working with the data: it is the glue between the RDBMS and all other aspects of a functioning software, i.e. the GUI, any kinds of APIs in the modern world, other side effects, and so on and forth. And *those* things are not something that shout "do me like SQL does".

Inventing such a general language would be a huge undertaking, and it would be very unclear what the benefit would be. It would very likely end up in the big heap of interesting, fun, or even powerful languages (like many functional or logical languages) that see very little usage in the real world. It would also have to be very tightly coupled to the RDBMS of choice to make any sense, so even if a company like Oracle would decide to invent such a thing, it would be a fringe language used just for that RDBMS.

Finally, the question of how to work with RDBMS from a modern programing language has been solved long ago. These days, all major languages have well-established frameworks that make it as painless as you want it to be. There just is no need.
> **Comments:**
> **username_0** (2024-12-17): *"that make it as painless as you want it to be" - (roaring laughter).  
> **username_36** (2024-12-25): I'm happy to have sent a smile your way, @Steve. ;)  
---
### Answer by username_34 on 2024-12-18. Score: 1
The reason why SQL has not been adapted as a full stack language is because it is frankly, rather awful as a language.

Several things to consider: OP asked about the SQL language, not the underlying RDMS system that it connects to. This is a lot of complex logic to impliment ACID principals on a specific set of hardware.

I think the underlying question was really asking,'I like SQL syntax, why can't I just use it everywhere?'

SQL is written with the assumption that you are largely performing set operations on data to transform it. The 'how' things are done is abstracted away in SQL, but in a lot of cases, this sort of detail is actually rather important.

Most coding problems are just simpler to work with under a model of imperative commands. SQL has the ability to run in an imperitive fashion, if you want to dive into cursors (but, dont). However, these tend to be very slow and difficult because SQL is optimized for set like operations.

But back to the The SQL language - it isn't very good as a 'programming' lanuage. It is internally inconsistent, and it lacks many basic features essential to work as a universal coding tool. Unless you are using a vendor extended version, it doesn't even have variable support. It is intended to basically be a stateless scripting language. It might be turing complete, but you would have to use the underlying RDMS to make that work. It would need a lot of extension to become a univeraslly useful language, and then it wouldn't probably look much like SQL anymore.

I am guessing that the OP is a new(ish) coder who has not explored other languages yet. Go try some out, there is a richness in diversity. Check out Lisp, that has set like operations and can be used to build actual apps.
> **Comments:**
> **username_0** (2024-12-18): Unfortunately, it wasn't about the syntax! You've also confirmed a thesis I mulled earlier, that part of my problem was that I didn't succeed in telegraphing the depth of my expertise!  
> **username_34** (2024-12-19): If it isn't about the syntax then, I would double down on my suggestion to look at Lisp or some of the other languages that treat data in a set like fashion. That might be the closest thing to what you are looing for.

Or if you are a coding expert, go invent S++, the full stack, turing complete version of SQL.  
> **username_0** (2024-12-19): Thanks Roger, but Lisp is not the sort of language I was thinking about - not because it is not a powerful language, but because there is no connection to SQL in the design. I'm very sceptical that two languages designed independently would be found to marry up well, and there's no substantiation that they do marry. I did a brief search for some kind of connection, and the horror of doing SQL-like things in Lisp can be seen here for example: https://mattpo.pe/posts/sql-but-lisp/. Also I tend to be more sympathetic to procedural languages, especially with ALGOL-style syntaxes. (1/2)  
> **username_0** (2024-12-19): Regards inventing programming languages, I tend to think that useful languages are at least the product of a collective of people whose full-time employment is concerned with design, and whose minds are properly oriented towards the various industrial developer constituencies (rather than the fancies of academics). Think IBM or Bell Labs, or Grace Hopper in the US Navy. I know the remark wasn't intended in the following manner, and the underlying issue here is just a deep disconnect, but in telling me to go invent my own, you may as well be telling me flippantly to "go play in traffic"! (2/2)  
> **username_27** (2024-12-20): If you're claiming to be a renowned civil engineer, it makes _some_ sense to say "go play in traffic" when you ask about a weird new crosswalk design. It doesn't excuse what could be considered attempted murder, but if you're looking for something weirdly specific that no one trying to answer your question is apparently getting at, why _not_ make it yourself?  
> **username_34** (2024-12-20): Well, I have no idea about what your goal is then. If you aren't in it for the syntax, or the set like data operations, I don't see what you are looking for. Putting things in abstracted storage layers?

Remark about inventing it yourself wasn't flippant. Lots of brilliant things were made by engineers who wanted something that nobody else was building. If you want something that doesnt exist, go build it. Good luck!  
> **username_0** (2024-12-20): @Someone, because I thought there would be more people here living in the real world where the question I've asked - about the reconciliation of the world's premiere database language, and other languages that seemingly always have to be used to interact with it - is not "weirdly specific".  
> **username_0** (2024-12-20): @RogerHill, so I've explained to you that it came across flippant, and you've just repeated the same line, without taking stock of why I might have thought that? The vast majority of useful things are made by legions of paid professionals. Have you checked whether I'm leading such a legion, or that I'm even engaged in language design professionally, where it might be feasible for me to follow your encouragement? (1/2)  
> **username_0** (2024-12-20): Secondly, since you admit you've no idea what I'm on about, why would it be sensible for you to encourage me to do something you already suspect you don't even remotely understand? Before I proceed to explain my goal further, do you regard yourself as a fair expert on the SQL language - is it something you know deeply as a theoretician with a special interest in the area (for example, are you familiar with the many differences between RA and SQL?) and/or do you use SQL (or have used) as a professional expert on the language? (2/2)  
> **username_27** (2024-12-20): I feel like one doesn't have to be an expert to give an opinion. The response was basically "Clearly, no one else here is giving you language examples you want, so if you want/need one, why not just make it yourself?"  
> **username_2** (2024-12-21): @Steve The entire premise of this site is that languages don't need to be designed and implemented by unique geniuses. To take umbrage with the suggestion that you design and implement your own language, on a site where the basic assumption is that people here are interested in designing and implementing languages, is like going to a ski resort and then taking umbrage with the suggestion to go skiing.  
> **username_2** (2024-12-21): Also, it is a bit rich to accuse people of being inflammatory one sentence after you labelled other users on this site "silly people".  
> **username_0** (2024-12-21): @kaya3, I don't agree. The basic assumption is that people are here to exchange information about language design, not to engage single-handedly (or at all) in the design of a new language, and that is not even the way useful languages are normally designed.  
> **username_27** (2024-12-22): Not all languages have to be ubiquitous or "useful" to everyone. If your specific need is a language that can interface between SQL's jankiness and other languages on the development stack, there's no problem with designing a small interface language. Even if you're the only user, there is still _a_ user, which means the language is, by definition, "useful".  
> **username_0** (2024-12-22): @Someone, it seems that you're proceeding from assumptions very different from my own, as well as continuing to misinterpret the question. Firstly, my need wasn't for a third language to interface between two existing languages, SQL on the server-side and something else on the client-side - it was to purge the usage of languages on the client-side which are conceptually incommensurate with SQL. To have a third language which "translates" but does not purge anything, would only add to the deadweight - I'm not sure whether you're in touch with what I mean by this "weight". (1/2)  
> **username_0** (2024-12-22): Secondly, the "usefulness" has to be weighed against the development and maintenance cost of not just the language itself, but the development environment and the integration into a toolchain, and the reproduction of knowledge about all these things into colleagues' minds. Something that demands more labour than it saves is not "useful". And aside from whether the competencies exist, you don't seem to appreciate the amount of labour this would entail. You don't seem to appreciate the extent to which your suggested actions are extreme and unheard-of. (2/2)  
> **username_27** (2024-12-23): Considering how scuffed I've heard some large companies' _production code_ is, it doesn't seem _completely_ outlandish to have some company develop an internal "special sauce" client language that interfaces with the SQL layer. But it would be quite difficult, yes, which is probably part of the reason why no one does it.  
> **username_27** (2024-12-23): But also, I feel like client-side code shouldn't rely too much on the internal DB's structure—it makes more sense to be modular with one's architecture. If the company were to change the structure (for example, creating a dedicated security layer between the database and the client), it would completely invalidate that "special sauce" that was supposed to interface between the client proper and the database.  
> **username_0** (2024-12-23): @Someone, so we are agreed that the suggestion that a sole developer write their own language was a foolhardy suggestion?  
---
