Skip to content

refactor: sort/dedup lineage nodes once, not per-node#6084

Merged
max-sixty merged 1 commit into
mainfrom
refactor/collect-frames-sort-once
Jul 18, 2026
Merged

refactor: sort/dedup lineage nodes once, not per-node#6084
max-sixty merged 1 commit into
mainfrom
refactor/collect-frames-sort-once

Conversation

@prql-bot

Copy link
Copy Markdown
Collaborator

Problem

FrameCollector::fold_expr (in semantic/reporting.rs) is invoked once per expression node while building the debug-lineage node graph. On every one of those calls it re-ran:

self.nodes.sort_by_key(|a| a.id);
self.nodes.dedup();

Since fold_expr runs n times and each call sorts the growing nodes vector, collection was O(n² log n) for a query with n nodes — pure wasted work that scales badly on large queries. Nothing reads self.nodes mid-traversal, so the intermediate sorts served no purpose; only the final ordered/deduped vector is used (by collect_frames, for the parent-linking pass).

Solution

Move the sort + dedup out of the per-node fold_expr and into collect_frames, running them once after the full traversal completes (mirroring how collect_frames already does frames.reverse() and parent-linking after folding). This drops the work to a single O(n log n) sort. The final nodes vector is byte-for-byte identical, because a single sort+dedup after all pushes produces the same canonical result the repeated in-loop sorts converged to.

Testing

This is a behavior-preserving change, so there is no "fails-before / passes-after" regression test to add — output is unchanged by construction. Correctness is guarded by the existing debug_lineage integration snapshots, which serialize the full node graph that collect_frames produces:

  • All 31 queries::debug_lineage::* snapshot tests pass unchanged.
  • The reporting/lineage lib tests pass.

Found during the nightly code-quality survey.

FrameCollector::fold_expr sorted and deduped the entire nodes vector on
every node visit, making debug-lineage collection O(n² log n). Move the
sort/dedup to run once after the full traversal in collect_frames; the
result is identical since nothing reads nodes mid-traversal.
@max-sixty
max-sixty merged commit f9cc67a into main Jul 18, 2026
36 checks passed
@max-sixty
max-sixty deleted the refactor/collect-frames-sort-once branch July 18, 2026 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants