SQLite not open enough and needs modernization complains project which forks it

SQLite not open enough and needs modernization complains project which forks it

A new fork of SQLite, called libSQL, aims to modernise this massively popular embedded database, with its founder complaining that “SQLite is explicitly and unequivocally “Open Source, not Open Contribution.”

Glauber Costa is founder and CEO of ChiselStrike, whose product generates back-end applications from TypeScript classes. In what he calls the manifesto for libSQL, he and his team argue that SQLite needs extending with critical new features. These include:

  • Support for distributed databases available across multiple servers
  • Support for asynchronous I/O via the new Linux io_uring API
  • Using eBPF to optimize SQLite by allowing some operations to run in the kernel
  • Adding support for user-defined functions in Wasm (WebAssembly) in order to allow other languages, which compile to Wasm, to be used instead of C.

The libSQL project plans to use Rust alongside C to implement these changes.

Why a fork though, and not making proposals to the SQLite project itself? The issue is that the SQLite project has a narrow focus to be a fast, small and reliable embedded database. The code is open source but it is not a public project in the same way as most other well-known open source applications, including Linux itself. The original author Dr D Richard Hipp still leads the project. He told me in 2007 that “Our goal is not to add all sorts of bells and whistles, but rather to keep SQLite small and fast.” He added: “Other people are free to make a copy of SQLite and fork the tree if they want to … it won’t hurt my feelings. But it has turned into a very complicated piece of software, and it takes a dedicated effort to keep it in good shape.”

In response to today’s news, a developer on Hacker News commented that “SQLite is as close to the gold standard of software quality that we have in the open source world. SQLite is the only program that I’ve ever used that reliably gets better with every release and never regresses. Could it be that this is precisely because they don’t use ‘modern tools’ and accept outside contributions?”

It is a hard case to prove but the SQLite project does demonstrate that open source can work well with models that are quite different than the community collaboration that has become common. SQLite is built into so many systems, not least Apple’s macOS and iOS, and Google’s Android, that is is probably the most widely used relational database engine.

As Costa himself notes, the success of SQLite combined with its relatively closed development makes it difficult for a fork to succeed. Another option is to wrap SQLite with additional functionality but this has limitations, which Costa alludes to. The libSQL team add that “ff and when SQLite changes its policy to accept contributions, we will gladly merge our work back into the core product and continue in that space.”