Code to disable Python’s Golal Interpreter Lock (GIL) was this week merged into python:main, progressing a major change to CPython, the reference implementation of the language.

The pull request adds support for PYTHON_GIL=0 or -X gil=0 when running the Python interpreter, provided it was compiled with free threading. 

The related PEP (Python Enhancement Proposal) is 703, making the GIL optional in CPython. This was accepted by the steering council for release in 3.13, but “with the clear provisio: that the rollout be gradual and break as little as possible, and that we can roll back any changes that turn out to be too disruptive.”

The GIL is an obstacle to concurrency since it prevents multiple threads from running code at the same time. For example, the PEP quotes an engineer at DeepMind saying, “in many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer than 10 threads the GIL becomes the bottleneck.” The workaround is to put such code into native code modules but this “makes the code less accessible to researchers.”

There is also reference to the GIL making code to deploy AI models more difficult, a key topic considering the explosion of interest in AI software.

It is expected that disabling the GIL will break some existing code. “We’re expecting disabling the GIL to be pretty buggy initially. So even if we’re targeting disabling the GIL by default in 3.13, we’ll probably want the GIL to stay on by default for a while,” said Python member Brett Simmers. 

One of the immediate issues, said Simmers, is that “With PYTHON_GIL=0 set … trying to run the full test suite crashes pretty quickly, in test_asyncio.”

Meta engineer Sam Gross, the prime mover behind the current proposal, said: “We should keep the GIL enabled by default in free-threaded builds (as it is currently is) for now … we can add the Py_mod_gil slot and eventually disable the GIL by default later on, but hopefully still before the 3.13 release.”

Work is also under way to support extensions that require the GIL. The Py_mod_gil slot referenced above is a class variable that can be set to indicate that the GIL is not required, but “If the slot is not set, the interpreter pauses all threads and enables the GIL before continuing,” according to PEP 703. Since this can slow performance, the interpreter will also issue a warning in these cases.

Python version 3.13, which will include the code to disable the GIL, is scheduled for release on October 1, 2024.