Turso, a developer focused on database solutions, is re-implementing the SQLite database engine in Rust, in order to facilitate major new features such as asynchronous I/O.

Turso was founded in 2021 by Glauber Costa and Pekka Enberg. In 2022 the startup, then called ChiselStrike, forked SQLite to create libSQL. The rationale was that SQLite needed modernization, and that while the project is open source, it is not open to contributors who wish to enhance it.

The founders now assert that merely forking SQLite was insufficient for some of the changes they wanted to make – including asynchronous I/O. Enberg started a private project to re-implement SQLite in Rust, and this has been adopted as an official Turso project.

The intention is that Limbo (apparently a codename and likely to be changed) will be fully compatible with SQLite, but with “full memory safety and on a new, modern architecture.” The project is currently a work in progress and is on GitHub under the MIT license.

The reliability of SQLite is partly thanks to its extensive suite of tests. Costa and Enberg state that they can achieve the same or better reliability with a technique called Deterministic Simulation Testing (DST), with the assistance of another dev – antithesis, specialist in testing software in an environment that simulates hardware and software failures. Limbo also has its own DST framework built-in.

Limbo on Linux uses the kernel system call io_uring, developed by an engineer at Meta for high performance asynchronous I/O. The new engine extends sqlite3_step to return immediately if data is not ready to consume. Although SQLite queries are generally fast, they can be slow with large datasets, Costa and Enberg observed.

Another Limbo feature is compilation to web assembly (WASM), which the Turso co-founders said was “an afterthought for SQLite.” 

Performance of Limbo is already good – faster than SQLite in some cases, slower in others. “The goal here is not to claim that it is faster, though (it isn’t) … it is to highlight that we already reached a good level of performance this early in the project,” wrote Costa in a comment on Hacker News.

We were able to compile and run Limbo on a Mac, though we needed to update the Rust compiler to the latest version to overcome errors. 

Is reimplementing SQLite in Rust a worthwhile project? “Given the code quality and rigid testing, SQLite is probably the last project that should be rewritten,” remarked one developer. That said, the purpose of the rewrite is not to avoid vulnerabilities common in C or C++ code, but to add features – given that the SQLite project itself is relatively closed – and to take advantage of the Rust language for things that are hard to achieve in C.