Google researchers suggest “fixing” SQL with pipe syntax, SQLite creator unconvinced

Google researchers suggest “fixing” SQL with pipe syntax, SQLite creator unconvinced

Google says it can improve SQL by introducing a pipe operator – which is already in use internally – but Dr D Richard Hipp, creator of the SQLite database, is unconvinced by the plan despite implementing it in an experimental build.

In a paper entitled “SQL has problems. We can fix them: pipe syntax in SQL,” a group of 13 Google researchers argue that “SQL is an old language with significant design problems, making it difficult to learn, difficult to use, and difficult to extend.”

One of the oddities of SQL is that queries begin with the SELECT clause but the validity of this depends on the FROM clause which appears after it. It would be more logical to have the FROM clause first. The researchers want to replace the “rigidly defined order” of SQL with pipe syntax that allows any order. The pipe character is ‘|>’ rather than the more natural ‘|’, in order to avoid ambiguity with the SQL bitwise OR operator, also ‘|’.

The researchers identify numerous issues with standard SQL, including not only the clause order, but also duplicate clauses, need for subqueries, and that Common Table Expressions (CTEs), or temporary named result sets, are too verbose.

Google has already implemented pipe syntax internally, as a feature of GoogleSQL, the dialect of SQL used for database managers including BigQuery. According to the paper, the company is now “exploring supporting pipe syntax externally in BigQuery and Spanner.”

Google is a big player and if it goes ahead with implementing the pipe operator for external users it could become widely used, but in a discussion on the SQLite forum, Hipp said that “I do not like the pipe operator … the authors list 11 reasons why they think the pipe operator is useful in section 4.1.4. I remain unconvinced.” He did acknowledge though that in one of the examples, “FROM-first better matches the semantics of the statement.”

A short time later, Hipp posted an experimental quick prototype of SQLite online that supports some of the proposed pipe syntax. SQLite is perhaps the most widely used relational database, though most often at small scale, and Hipp said that “my goal is to keep SQLite relevant and viable through the year 2050” though he does not wish to add non-standard extensions. He said he would take his cue from PostgreSQL. “If PostgreSQL adds support for FROM-clause-first queries, then I’ll do the same with SQLite,” he wrote.

Nevertheless, he said he lost interest in the Google pipe syntax after investigating his own use cases, for example in the Fossil version control system which he created. He likes the idea of FROM-first syntax, but not the pipe syntax.

A comment from a developer on Hacker News said that “FROM first would be nothing short of incredible. I can only hope that Postgres and others can find it within themselves to get together and standardize on such an extension.”