Kill the GIL?¶
See the Global Interpreter Lock.
Why does CPython need a global lock?¶
Incomplete list:
- Python memory allocation is not thread safe (it should be easy to make it thread safe)
- The reference counter of each object is protected by the GIL.
- CPython has a lot of global C variables. Examples:
interp
is a structure which contains variables of the Python interpreter: modules, list of Python threads, builtins, etc.int
singletons (-5..255)str
singletons (Python 3: latin1 characters)
- Some third party C libraries and even functions the C standard library are not thread safe: the GIL works around this limitation.
Kill the GIL¶
- Require deep changes of CPython code
- The current Python C API is too specific to CPython implementation details: need a new API. Maybe the stable ABI?
- Modify third party modules to use the stable ABI to avoid relying on CPython implementation details like reference couting
- Replace reference counting with something else? Atomic operations?
- Use finer locks on some specific operations (release the GIL)? like operations on builtin types which don’t need to execute arbitrary Python code. Counter example: dict where keys are objects different than int and str.
See also pyparallel.