A Short Story About SQL’s Biggest Rival

October 9, 2020 7:21 PM

“Maybe QUEL is better than SQL. Maybe French is better than English? It didn’t matter: English and SQL were going to win.” Oracle Version 4 was a good product — certainly better than Oracle Version 3, which was released to the market with more bugs than a discarded pomelo. But it didn’t win because it was technically superior to Ingres. It won because IBM was powerful, and because Stonebraker made a mistake.
posted by geoff. (43 comments total)

And that little fluke in relational database history grew up to be Post-Ingres SQL, PostgreSQL.
posted by geoff. at 7:37 PM on October 9, 2020 [2 favorites]

I wish I lived in a world where QUEL won. Bleh.
posted by Mister Cheese at 7:38 PM on October 9, 2020 [1 favorite]

I cannot wait to read this thank you for posting this
posted by capnsue at 7:39 PM on October 9, 2020

I would have liked to have more detail about QUEL, both because it’s an interesting what-if scenario and because the two analogies which the author drew (Dvorak keyboards and Esperanto) are notoriously oversold by their proponents and that would certainly match the image I’ve developed of Stonebraker being prone to overestimate the degree to which technical elegance in one area dominates a complex product comparison. The examples on Wikipedia are interesting but I’m not seeing an obvious productivity boost.
posted by adamsc at 8:55 PM on October 9, 2020 [11 favorites]

I spent years thinking SQL was pronounced “squirrel”. It’s still the first thing I hear when I read it.
posted by q*ben at 9:08 PM on October 9, 2020 [4 favorites]

To be pedantic, PostgreSQL was a fork of Postgres. I don't recall if Postgres still used QUEL, but it originally didn't use SQL, like many competing database packages of the era.

It took a while for SQL to drive out everything else. At the time, it seemed like MySQL that finally closed the door, despite its completely nonstandard dialect. To be fair, PostgreSQL also had plenty of its own idiosyncratic syntax and spotty coverage of language features in the early days.
posted by wierdo at 9:35 PM on October 9, 2020

I worked in IT maybe right at a decade. I worked in a plug compatible (plug compatible meaning nothing more than it was pretty much straight-up a copy of IBMs big iron; the saying was that if you put a Hitachi coffee cup on your desk you'd save a million dollars when IBM came calling to upgrade or up-sell your mainframe) mainframe shop,

I never could get my head around *any* database, hierarchical or relational, not really. Had I stayed in IT it's almost a certainty that I would have figured it out, as familiarity does -- in time -- familiarity does work its way upon me. As it was, the best thing I knew about database code was how to copy it from a co-workers library and stuff it into my code -- bingo, look at that, it works great.

For whatever reason (or reasons) tables (arrays probably in your world) tables came to me really well, I could easily keep a lot of balls in the air in my mind when writing tables, I actually enjoyed digging into tables while many others would have razors poised over their wrists, tears coursing down their cheeks. Which is how I felt when trying to make database code happen.

So I have a real respect for those of you who can not only keep track of where you are inside a database but also to do it easily. It was frustrating as hell for me to work with you, those of you who made perfect sense of it. It's a gift, one that I was not given.
posted by dancestoblue at 10:04 PM on October 9, 2020 [4 favorites]

I thought Ingres grew up to be Pokémon Go.
posted by bjrubble at 10:48 PM on October 9, 2020 [10 favorites]

Without PostgreSQL would we have PostGIS? Maybe, maybe not but I shudder to think...
posted by klanawa at 12:53 AM on October 10, 2020

I spent years thinking SQL was pronounced “squirrel”. It’s still the first thing I hear when I read it.

UP.gif
posted by axiom at 2:52 AM on October 10, 2020 [1 favorite]

Is there any technical reason that QUEL couldn't be added as a second query language for an SQL database? Or does it depend on a different underlying relational model?
posted by clawsoon at 3:48 AM on October 10, 2020

Oracle Version 4 was a good product

[citation needed]

Sorry. I just have a thing about Oracle and its clunky, overly-complex, don't-work-like-anything-else installation and configuration procedures. Ask me some time about installing an Oracle database driver for Python versus a PostgreSQL one.

(In contrast with dancestoblue, I'm one of the database guys as part of my job. Specifically, I'm the one people come to for writing and running queries. The high/low point of it all so far has been this forty-line monstrosity that was packed full of subselects, JOINs, and CASE WHENs. It works, but it's not pretty.)
posted by Mr. Bad Example at 3:59 AM on October 10, 2020 [4 favorites]

The Wikipedia page has some examples comparing the two languages, and the UCB Database Research Group still hosts the code for University POSTGRES. "As of 1994, this software is no longer being developed or supported."

And so it was that the engineers who created SQL were focused on database performance, not language design, and they never expected the user interface they invented to take off and become a standard.

Every time I decide on a set of input element names and values for a collection of related HTML forms I get the vague sense that I'm creating a query language, and probably not a very good one. I wonder if some of the SQL engineers felt the same way.
posted by clawsoon at 4:00 AM on October 10, 2020 [2 favorites]

QUEL versus SQL: one more for worse-is-better*?

*: While searching for these articles it turns out that worse-is-better is... you decide.
posted by k3ninho at 4:42 AM on October 10, 2020

I think that, if you assume that there's a significant number of people working with databases who aren't traditional developers or sysadmins or whatever, any database language that's close to human-readable is likely to win out. SQL (at least for simple queries) is often understandable at first glance. You don't need much prior knowledge beyond understanding the concept of a field and a table. However, SQL can become an utter nightmare as soon as start combining a load of joins, counts and sums into gigantic single queries.
posted by pipeski at 4:59 AM on October 10, 2020 [2 favorites]

It was also not clear at that time if relational db's would be successful, they were slow compared to well optimized databases and limited in size. One of IBM's clever marketing phrases at the introduction of Db2 was that it "could hold so much data that you could not hit all of it with a firehose" - someone had calculated the total number of disk drives (which were pretty big at the time) that would be needed to hold the theoretical maximum amount of data the code could handle and it was bigger than a football field or something.
posted by sammyo at 5:57 AM on October 10, 2020 [2 favorites]

Whoa, completely oblivious about this. SQL's syntax seems cleaner to me but its syntax has is still frustrating, so much so that I try to avoid databases as much as possible and instead use json for small web projects, which is its own kind of insanity.
posted by Foci for Analysis at 7:14 AM on October 10, 2020

Stonebraker, Michael; Wong, Eugene; Kreps, Peter; Held, Gerald (1976). "The Design and Implementation of INGRES": https://people.eecs.berkeley.edu/~wong/wong_pubs/wong46.pdf

A perhaps more illustrative query from that paper:

RANGE OF E, M IS EMPLOYEE 
RANGE OF D IS DEPT 
RETRIEVE (E.NAME) 
  WHERE ESALARY > MSALARY AND 
        E.MANAGER = M.NAME AND 
        E.DEPT = D.DEPT AND 
        D.FLOOR# = 1 AND 
        E.AGE > 40

posted by Monday, stony Monday at 8:08 AM on October 10, 2020 [3 favorites]

(the translation to sql is left as an exercise to the reader; I took my DB course in... 2003 I think? and haven't really touched sql since)
posted by Monday, stony Monday at 8:15 AM on October 10, 2020

Very interesting, thanks.

Career DBA (going on .. oof.. 28 years now) for .. yea, basically all the things. Plus code. So this hit me right in the wheelhouse :)

SQL quirks between products are the bane of my existence - it's a constant headache codeswitching between them (literally! Heh).

I too would have liked to know more about Quel! Monday, stony Monday's link & excerpt gave me a little furrow in my brow though. It's bizarrely half-familiar, partially from the operations that SQL does *almost* the same through to stuff you'd find in code these days. I agree though that SQL is simply more usable/easy to get started with - Quel has a whiff of elitist engineerism about it. Databases are at the heart of absolutely everything these days, I wonder if that would still have been true if access to the data was only through an abstruse engineering language... (Probably. I mean, people apparently use Haskell. What do I know.)

Most of the trouble people get into with SQL are from trying to make it do too much - basically trying to write application code in SQL. Trust me Mr. Bad Example, I would be delighted to encounter a mere forty-line query. Some of my canned DBA queries are more than that. Some of the godawful developer crap we end up having to tune is literally pages of unions, joins, subselects, all crawling with substrings and concats and other functional murder to indexes... nightmare. Databases are awesome at getting your data, FAST. SQL does that. Do the damn business logic elsewhere - either a) in sensible schema design, or b) in code. If your subselects and joins are piling up - you have a schema problem. Don't even get me started on generated queries - eg Tableau. Ugh.

Random observation - I have found that those who started on DB2 tend to call it "Ess Queue L", and the Oracle tribe tend to call it "Sequel" - I am much amused to find out it was in fact originally an IBM development and called Sequel for real. I guess IBM got the "argh no mustn't call it that" culture and nobody else did...
posted by Ilira at 9:50 AM on October 10, 2020 [6 favorites]

(In contrast with dancestoblue, I'm one of the database guys as part of my job. Specifically, I'm the one people come to for writing and running queries. The high/low point of it all so far has been this forty-line monstrosity that was packed full of subselects, JOINs, and CASE WHENs. It works, but it's not pretty.)

I actually really like writing SQL (as a dev, not formally a DB specialist). Was working at a place where there was probably a lot more of that going on manually than necessary and then just joined a team that’s all ORM and honestly I miss it.
posted by atoxyl at 10:20 AM on October 10, 2020 [1 favorite]

Does anyone with more database experience know whether this all matters? I have used ORMs and data providers (query languages?) like LINQ for ages, across platforms and try to use the one that hides the implementation details as much as possible. My high level understanding is that QUEL has better composability which leads to cleaner queries as the complexity increases. Is this correct? The paper vaguely talks about relying on functions versus predicate calculus. I guess my math skills have wained as I don't know or remember why this is better than something like LINQ which relies on predicate calculus.

In any case in my day to day, rarely is data retrieval the bottleneck and I use the most natural query method possible to extract what I want. I would prefer that at the program level the code is as expressive as possible and the plumbing is hidden. Also, it is way easier to get developers familiar with Java/C#/Javascript and tell them to write clean expressive code to let an ORM take care of it then it is to tell them to write things in an obscure query syntax they'll probably get wrong anyway.
posted by geoff. at 10:27 AM on October 10, 2020

something like LINQ which relies on predicate calculus.

That should read "lambda calculus"
posted by geoff. at 10:41 AM on October 10, 2020

My hot take is ... meh. I like the composability of QUEL, and it seems more natural to me, but it's just syntax. There's a lot more to a DBMS than just writing queries, and the query optimizer munches your original query beyond recognition anyway.
posted by RobotVoodooPower at 10:43 AM on October 10, 2020 [1 favorite]

I found a Powerpoint presentation on Oracle's early history, and it's so interesting. Oracle 4 was when they introduced read consistency, version 6 added B-tree indexes, version 7 added parallel query, and RAC (shared-nothing clusters) came in 2001. It's like speed-reading through a database textbook.

SQL gets truly horrible when it tries to cram OLAP / business-intelligence features into the language. I do like the more recent development of SQL support of json though.
posted by of strange foe at 11:58 AM on October 10, 2020 [1 favorite]

the translation to sql is left as an exercise to the reader

In SQL that's something like:

SELECT E.NAME
FROM EMPLOYEE E 
     JOIN EMPLOYEE M ON E.MANAGER = M.NAME
     JOIN DEPT D ON E.DEPT = D.DEPT
WHERE E.SALARY > M.SALARY AND 
      D.FLOOR# = 1 AND 
      E.AGE > 40

Not much different, but this isn't a particularly complicated query.
posted by axiom at 12:37 PM on October 10, 2020 [1 favorite]

Hmm; there's readability, writeability, and debuggability. Would QUEL have better error messages than SQL?

Or like everyone I know in my discipline, you rely on a bunch of canned queries that you check in and make variants of when you need something different.
posted by batter_my_heart at 2:01 PM on October 10, 2020

Would QUEL have better error messages than SQL?

I haven't read a SQL standard or anything, but I don't think that it specifies error messages, instead leaving that up to the DB software. At any rate, database implementations can screw up error codes all on their own.

I use Oracle, one fairly common error is ORA-00942: table or view does not exist, which is a bad error code because it doesn't tell you which table or view from your SQL statement (ostensibly) doesn't exist. It can be reported for nonintuitive reasons, for example, you name a table that does exist but for which you do not have read permission. It can also be thrown for completely mundane reasons, like you accidentally typoed a table name in the 12th join clause out of 20 and good luck finding that!
posted by axiom at 2:22 PM on October 10, 2020

I can write SQL but I really have no knowledge of database theory. That probably makes me a shit developer but oh well.
posted by Young Kullervo at 4:52 PM on October 10, 2020

Not much different, but this isn't a particularly complicated query.

On PostgreSQL, one can use implicit joins to condense the syntax to be approximately the same verbosity (and read more obviously as equivalent) to the QUEL version. I suppose that shouldn't surprise me.

I'd get more worked up about non-standard SQL, but there has never been a time when DBMSes haven't had quirky implementations of the language, missing language features, and their own unique extensions. Many have improved lately to at least be able to parse standard syntax, but it still feels like tilting at windmills.
posted by wierdo at 5:10 PM on October 10, 2020

Cracks me up, but none of the younger folks I work with really know SQL at all.
posted by ph00dz at 5:53 PM on October 10, 2020 [1 favorite]

The history of software world is chock full of "objectively superior" technologies with creators and adherents that smugly sit back and wait for the entire world to come calling, except that "entire world" is too busy making money hand over fist to even give that "superior" technology the time of day.

I was a heavy raw SQL user from 1999 to 2009ish, and from then until about 2018 I used it via various ORMs (these days I'm way into world-wide eventually constant denormalized key value stores. Fun!). At no point did I feel SQL wasn't up to the task. In fact, as time went on and my skills and the available technologies became more robust, my SQL became simpler and simpler until I was practically at the promised land of 100% materialized views. "Here's this new thing that lets you do more complex shit" would not have been an effective selling point with me.

But, the main story here is that these guys went on the wrong side of IBM, in the 80s! In a situation where IBM was getting their way standardized! That was like playing Russian Roulette with six bullets. There was a saying "No one got fired buying IBM", well the corollary is: Lots and lots of people got fired for doing something different than IBM. There really isn't a contemporary example.
posted by sideshow at 9:31 PM on October 10, 2020 [3 favorites]

Between ORMs being good enough and (almost) always being able to pay Amazon or someone else a bit more money if you aren't getting the necessary performance, we're past the point where most people need to write optimal SQL. Hell, now that SSDs are big enough, most people don't even need to bother with good indexing. Add to that the reincarnation of key-value stores as the new hotness, and I'm not the least bit surprised people aren't bothering to learn SQL any more.

It makes me sad to think of all the wasted energy in the form of unnecessary CPU cycles and disk accesses, but people long ago decided that reduced developer hours are worth a higher electric bill and the resulting extra carbon emissions.
posted by wierdo at 9:54 PM on October 10, 2020

SQL Programmers be like
posted by TheophileEscargot at 7:04 AM on October 11, 2020 [1 favorite]

Cracks me up, but none of the younger folks I work with really know SQL at all.
I see a *lot* of application databases done in rdbms that - besides the primary key - are completely unconstrained denormalized (flat) tables.

All the relationships are buried in application code in just-as-indecipherable invocations of twelve to twenty chained linq expressions.

Inferred constraints only knowable with psi powers and access to the jerk that wrote it.

I'd rather be debugging long, complex SQL queries, tbh.
posted by j_curiouser at 9:34 AM on October 11, 2020 [2 favorites]

From my standpoint as a DBA/Designer/Code Monkey back in those days, Oracle 4 and 5 were still spectacularly craptastic. I lost many hours of sleep to “ORA-0600: Internal Database Error” back in those days any time we ran sufficiently complex or large queries, most of which ran overnight. What Oracle got right in those days wasn’t so much the technology, but the sales side. They would make insane deals to convert people, figuring they’d get it back in the end. They had an army of PhD guys who made pitches to senior managers that were so compelling, even I sometimes forgot it was all bullshit in the middle of them.

The syntax differences between query languages back then were crucial, because the “optimizers” were anything but and you had to really carefully craft your syntax to ensure that the data was retrieved in a clever way. The resulting SQL was super-ugly because you had to do stuff like title||’’ = “Manager” to make sure that the engine wouldn’t use an index for a query that was going to retrieve half the rows.
posted by Lame_username at 9:41 AM on October 11, 2020

I see a *lot* of application databases done in rdbms that - besides the primary key - are completely unconstrained denormalized (flat) tables.

Thinking about it, this is probably the biggest reason I’m skeptical of the author’s take. If you use SQL written by people who learned their tools – or who use something like Django which was designed to encourage proper usage, it’s rare that SQL will be a limiting factor for most apps. The problems are when people were too busy to learn or architect and are avoiding cleaning up early mistakes, and I have no reason to think that dynamic would be different using QUEL. The NoSQL movement produced some neat ideas but mostly I’ve seen it produce the same unstructured mess with more bugs and reliability issues since there’s even more you have to get right.
posted by adamsc at 10:01 AM on October 11, 2020 [2 favorites]

I get that my side has lost, but I kind of hate ORMs. It seems like a whole lot of trouble in the service of, to me, the increasingly dubious goal of insisting that everything is OOP. The whole mentality is a catastrophe. The initiation beat-down is to pretend that "the n+1 select problem" is some kind of fact of nature that only clever and sustained attention to the arcane modelling needs of forcing a tight correspondence between database tables and ORM-controlled "domain classes" can deliver you from, rather than being a grotesque consequence of the catastrophic mistake you have made in selecting a tool that is in charge of delivering the jungle to your application whenever it asks for a banana. I suppose that by now the alternatives are no better, however, although I have wondered how bad it would actually be to just keep stored procedures for everything and send them sanitized inputs.
posted by thelonius at 10:27 AM on October 11, 2020 [2 favorites]

the increasingly dubious goal of insisting that everything is OOP

Counterpoint: if SOLID principles and encapsulation were used consistently, side effects would be a non-issue.

Immutability is an end run for devs who don't know how to, or refuse to write code without public accessors. The problem is, people think they can ignore encapsulation and still call it OODD.

All the 'functional' code I review just looks like procedural libraries from the 90s. It doesn't even pass the smell test for the aims of a functional idiom. It's a bunch of jokers refusing to write OODD *and* refusing to write uniformly *true* functional code, and ending up with procedural dreck.

(this is generally true at the last few shops I've been in. Certainly there are teams effectively implementing a functional paradigm - somewhere).
/rant
posted by j_curiouser at 12:44 PM on October 11, 2020

[orm] a tool that is in charge of delivering the jungle to your application whenever it asks for a banana

True that. Sometimes configuration helps, but jesus, default behavior is bonkers.
posted by j_curiouser at 2:30 PM on October 11, 2020 [2 favorites]

I admit it is very convenient once you finally do get it working right
posted by thelonius at 3:44 PM on October 11, 2020 [1 favorite]

Though I am late to this I have observations.

First I note an example of the writer showing that he is not as smart as he thinks he is:

But alas, that isn’t how the world works. If the world worked differently, we wouldn’t still be writing on QWERTY keyboards, or speaking English; technically superior alternatives like Dvorak and Esperanto would have taken over.

I was working as a developer when the action in the story took place but was way far away at the other end, doing projects where I might use a logic analyzer as a debugging tool, and later going so high-level as to write a kernel driver for the resulting monstrosity, followed by a data-acquisition app. I was aware of these packages, and that they were driving out dBase for the sort of people who wrote boring businessy shit, but they didn't seem relevant to cool rocket science (not giving much thought to what their innards were like).

I was able to remain blissfully ignorant of database technology for a long time. At length I got tired of the frequent job searches that result from working only on cool stuff and wound up owning (eventually) a big piece of a highly-SQL-dependent Windows thick-client app.

I have never tried to catalog the irksomenesses, but to my eyes (writing as one who in his youth collected programming languages like baseball cards) SQL is a train wreck. I like the remark in TFA, where it's called "not very composable." It looks like the outcome where you have a bunch of people doing implementation, and alpha users coming at them with "it would be nice if it could do X" and the devs saying "sure!" and proceeding to implement X directly, instead of figuring out how to make a set of consistent primitives where you can easily express X (and also Y , and, when the users get around to asking for it, Z).

The phenomenon of the technically inferior (or "wrong") option winning in the market has had huge influence on the hardware and software industries. The ascendency of Intel over Motorola in the CPU business (and really Intel design peripheral chips also), the dominance of MS-DOS over everything else, followed later by the dominance of Windows over alternatives that could clearly have been better, followed by the dominance of Microsoft's office apps over some initially-superior competitors... much of my world consists of suboptimal choices by the market.
posted by Aardvark Cheeselog at 7:44 AM on October 13, 2020 [5 favorites]

Yes, SQL is in fact a pretty poor language for general purpose programming. I suspect that is one of the reasons many modern RDBMSes allow you to write stored procedures in a wealth of other languages.

It is distressingly easy to be seduced into putting too much of an application's logic in the database, otherwise it wouldn't matter so much that the query language isn't a good programming language.
posted by wierdo at 8:39 PM on October 13, 2020