Crash in `DistributionFinder.context` because it is using `vars(self).update(kwargs)` under memory preasure

[sobolevn]

Crash report

Original report: https://gist.github.com/devdanzin/bf9fec4554c58c2a279b05b7ff8e6d9b (it produces the correct repro, but everything else is not correct, since it is AI generated and can be fuzzy).

Here's the real picture.

This class raises an error:

cpython/Lib/importlib/metadata/__init__.py

Lines 742 to 750 in 1785f4b

	class DistributionFinder(MetaPathFinder):
	"""
	A MetaPathFinder capable of discovering installed distributions.
	Custom providers should implement this interface in order to
	supply metadata.
	"""
	class Context:

Repro:

import faulthandler, importlib.metadata
faulthandler.enable()
from _testcapi import set_nomemory
for start in range(40):
    set_nomemory(start)
    try:
        importlib.metadata.metadata("f")
    except BaseException:
        pass
print("done, no crash")

Error:

Fatal Python error: _Py_Dealloc: Deallocator of type 'Context' cleared the current exception
Python runtime state: initialized

Stack (most recent call first):
  File "/Users/sobolev/Desktop/cpython/Lib/importlib/metadata/__init__.py", line 465 in from_name
  File "/Users/sobolev/Desktop/cpython/Lib/importlib/metadata/__init__.py", line 1091 in metadata
  File "/Users/sobolev/Desktop/cpython/ex.py", line 7 in <module>

Extension modules: _testcapi (total: 1)
[1]    14108 abort      ./python.exe ex.py

I tested this hack, no crash:

diff --git Lib/importlib/metadata/__init__.py Lib/importlib/metadata/__init__.py
index 32f4b7d2d6e..43ec93db749 100644
--- Lib/importlib/metadata/__init__.py
+++ Lib/importlib/metadata/__init__.py
@@ -777,7 +777,8 @@ class Context:
         """
 
         def __init__(self, **kwargs):
-            vars(self).update(kwargs)
+            for k, v in kwargs.items():
+                setattr(self, k, v)
 
         @property
         def path(self) -> list[str]:

But, this is clearly not a proper fix. Because for some reason the exception is cleared somewhere.

After:

» ./python.exe ex.py
object address  : 0x20002128190
object refcount : 3
object type     : 0x101267e70
object type name: MemoryError
object repr     : 
lost sys.stderr
object address  : 0x20002128110
object refcount : 3
object type     : 0x101267e70
object type name: MemoryError
object repr     : 
lost sys.stderr

[sobolevn]

Config: make clean && ./configure --with-pydebug --disable-gil && make -j$(sysctl -n hw.ncpu)
Simplier repro:

import faulthandler, importlib.metadata
faulthandler.enable()
from _testcapi import set_nomemory
import abc


class DistributionFinder:
    class Context:
        name = None
        def __init__(self, **kwargs):
            vars(self).update(kwargs)

    @abc.abstractmethod
    def find_distributions(self, context=Context()):
        pass

for start in range(40):
    set_nomemory(start)
    try:
        DistributionFinder.Context(name='f')
    except BaseException:
        pass
print("done, no crash")

Traceback:

done, no crash
Assertion failed: (_Py_IsOwnedByCurrentThread((PyObject *)mp) || IS_DICT_SHARED(mp)), function set_keys, file dictobject.c, line 206.
Fatal Python error: Aborted

<Cannot show all threads while the GIL is disabled>
Stack (most recent call first):

Current thread's C stack trace (most recent call first):
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at _Py_DumpStack+0x44 [0x1007b2234]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at faulthandler_dump_c_stack+0x68 [0x1007e19e0]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at faulthandler_fatal_error+0x1ac [0x1007e177c]
  Binary file "/usr/lib/system/libsystem_platform.dylib", at _sigtramp+0x38 [0x1a00d16a4]
  Binary file "/usr/lib/system/libsystem_pthread.dylib", at pthread_kill+0x128 [0x1a009788c]
  Binary file "/usr/lib/system/libsystem_c.dylib", at abort+0x7c [0x19ffa0a3c]
  Binary file "/usr/lib/system/libsystem_c.dylib", at err+0x0 [0x19ff9fc70]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at set_keys+0x70 [0x1004d5d3c]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at PyObject_ClearManagedDict+0x144 [0x1004d58ac]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at subtype_clear+0xd8 [0x1005630ec]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at delete_garbage+0x1c8 [0x10070ab20]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at gc_collect_internal+0x2ec [0x100709474]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at gc_collect_main+0x248 [0x100706708]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at _PyGC_CollectNoFail+0x20 [0x100706984]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at finalize_modules+0xc4 [0x1007658b0]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at _Py_Finalize+0x1c4 [0x100764bac]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at Py_FinalizeEx+0x14 [0x1007649e0]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at Py_RunMain+0x28 [0x1007cb5b8]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at pymain_main+0x64 [0x1007cba84]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at Py_BytesMain+0x38 [0x1007cbad4]
  Binary file "/Users/sobolev/Desktop/cpython/python.exe", at main+0x24 [0x1003588fc]
  Binary file "/usr/lib/dyld", at start+0x17bc [0x19fcf6b98]
[1]    14739 abort      ./python.exe ex.py

The crash seems different, but it might have the similar cause.