[simpleqt]

Summary

A key/value pair used to walk the table header of the current section from the
document root twice, for every pair:

• _parser.py:444: out.flags.is_(abs_key_parent, Flags.FROZEN)
• _parser.py:450: out.data.get_or_create_nest(abs_key_parent)

with abs_key_parent = header + key_parent (_parser.py:433). Flags.is_ and
NestedDict.get_or_create_nest both walk the path they are given from their
respective roots, so a section with a depth-D header and M keys is parsed
in O(M * D) time. MAX_KEY_PARTS bounds D at ~1000, but the
O(1000 * M) amplification remains.

This is gh-152930 — python/cpython#152930 — where a
CPython maintainer asked for the fix to be done in Tomli first.

Reproducer

import time
import tomli

D, M = 998, 40000
doc = "[" + ".".join(f"h{i}" for i in range(D)) + "]\n" + "".join(f"k{i}=1\n" for i in range(M))
start = time.perf_counter()
tomli.loads(doc)
print(len(doc), "bytes", time.perf_counter() - start, "s")

354 KB of valid TOML on this machine (CPython 3.11.14, Windows):

	time
master (43a86ad)	5.14 s
this PR	0.11 s

The header length alone no longer matters: with the same 5000 keys, a depth-1
and a depth-998 header now parse in 0.0134 s and 0.0133 s, where before the
deep header was 45x slower (0.0124 s vs 0.5587 s).

Fix

Resolve the section once, when its [header]/[[header]] line is parsed:
Flags.resolve() returns the section's position in the flags tree and
NestedDict.get_or_create_nest() is called for the header itself. loads()
keeps the pair and passes it to key_value_rule(), which then looks up
Flags.is_(key, flag, start=header_flags) and
get_or_create_nest(key, start=header_nest) relative to it.

Anchored lookups see the same flags as lookups from the document root: only
FROZEN is ever set recursively, a header can only be entered through the
table rules, which reject an already frozen header or one with a frozen
ancestor, and freezing always happens to keys that the current section header
is a proper prefix of. So neither the header nor its ancestors can become
frozen while the section is being parsed.

Tests

• test_many_keys_under_deep_table_header — a deep header with many keys
  parses into the expected tree.
• test_deep_table_header_not_rewalked_per_key — parses the same 5000-key body
  under a depth-1 and a depth-998 header, alternating between the two and
  keeping the fastest of five runs, then requires the ratio to be under 5.

Red on 43a86ad:

FAIL: test_deep_table_header_not_rewalked_per_key
AssertionError: 45.1563934767589 not less than 5 : deep header 0.5587s, shallow header 0.0124s

Green with this change (ratio 0.99): Ran 20 tests ... OK (skipped=1).

A ratio of the two timings is asserted, and both documents are measured
interleaved, so the test does not depend on how fast or how loaded the machine
is; before the fix the ratio is the header depth.

Verification

• python -m unittest: 20 tests, OK (1 skipped: the pure-Python 3.15+ lazy
  import test).
• Official toml-test suite, all 720
  case names in tests/files-toml-1.1.0 at 96fd088 (711 are valid UTF-8 and
  were parsed; 9 are intentionally non-UTF-8 decoding tests): no valid case
  rejected, no invalid case accepted — before and after the change.
• Parse results are identical to 43a86ad for 151,519 further documents:
  mutations of the corpus plus 104,976 exhaustive 4-statement combinations of
  table headers, array-of-tables headers, dotted keys, inline tables and frozen
  namespaces. Both the parsed value and, for rejected documents, msg, pos,
  lineno and colno were compared.
• coverage run -m unittest with tests/data/*/_external removed, like the CI
  coverage job: 100% statements and 100% branches.
• black 26.10.0, isort 9.0.2, flake8 7.4.1 (+ bugbear, builtins,
  comprehensions) and mypy 2.4.0 in strict mode: clean.

Not run locally: the mypyc wheel build and the other Python versions in the CI
matrix.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (43a86ad) to head (fe47054).

Additional details and impacted files

@@            Coverage Diff            @@
##            master      #327   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            4         4           
  Lines          535       550   +15     
  Branches       100       103    +3     
=========================================
+ Hits           535       550   +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.