Build name-tagger automaton as a noncontiguous NFA - #216
Conversation
Needles::build let AhoCorasickBuilder choose the automaton kind (Auto). Force NoncontiguousNFA: it is cheaper to build, which matters for the large needle sets the name tagger seeds (~470k aliases), whose one-time build dominates first-use latency. kind only changes the internal representation, not which matches are returned, and the prefilter stays enabled so search is unaffected for the short haystacks the tagger sees. Release build, person tagger (470,083 needles): Auto 1624 ms, ContiguousNFA 1516 ms, NoncontiguousNFA 1443 ms. About 11% off the automaton build; overlapping-search results are byte-identical across kinds. Existing test suite passes (192).
|
Same origin as #217 when every bit of cold start matters. |
pudo
left a comment
There was a problem hiding this comment.
Thanks for digging into the build cost — the 470k-needle figure and the kind comparison are useful data. But after a closer look I don't think the change is mergeable in its current shape, for three reasons:
1. The override hits every Needles consumer, not just the tagger. Needles::build also backs the three org_types replacers, whose find_iter runs on the per-comparison hot path (nomenklatura xref calls replace_org_types_compare once per candidate pair). The noncontiguous NFA is the slowest of the three representations at search time — sparse per-byte transitions plus fail-transition chasing, vs. the contiguous NFA that Auto selects for >100 patterns. For those consumers build time is irrelevant and search time is everything, so this trades a one-time ~180 ms saving for a cost paid millions of times in bulk workloads.
2. The safety argument in the comment doesn't hold. "The prefilter stays enabled ... search stays sub-microsecond either way" — as far as I can tell, no prefilter builds for these automatons at all: we set ascii_case_insensitive(true), which disables the memmem/packed prefilters, and the remaining start-byte/rare-byte prefilters require ≤3 distinct candidate start bytes, which a ~470k multi-script needle set (or the multi-language org-type sets) can't satisfy. So there's nothing mitigating the slower automaton walk, and the PR contains no search benchmark to support the "unchanged" claim.
3. Memory. The noncontiguous NFA is also the least compact representation, and the tagger automaton is cached for process lifetime — long-lived services (yente workers, crawlers) would hold that larger footprint permanently.
What I think would be mergeable: make the kind a per-consumer (or size/use-based) choice — noncontiguous NFA for the tagger, where build cost dominates and haystacks are short; Auto unchanged for the replacers — and include a search-side benchmark for the tagger path so the trade-off is measured rather than asserted. While you're in there, the module header's "~1k needles" premise (matcher.rs, design comment) should be updated or scoped to the org_types consumer, since it now contradicts the 470k figure by ~500×.
What
Needles::buildcurrently letsAhoCorasickBuilderpick the automaton kind (the defaultAhoCorasickKind::Auto). This forcesAhoCorasickKind::NoncontiguousNFA.Why
The name tagger seeds the
Needlesautomaton with ~470k aliases (mostly fromperson_names.txt). That automaton is built lazily on the firstanalyze_namescall with symbols enabled and cached for the process lifetime. On a cold process the build is the dominant startup cost, and the Aho-Corasick construction is ~86% of it.The default
Autokind does extra work selecting and constructing a contiguous NFA. The noncontiguous NFA is cheaper to build. For this workload build cost matters far more than search: the prefilter stays enabled and tagger haystacks are short (~30 chars), so search stays sub-microsecond either way.kindonly changes the internal representation, never which matches are returned, so output is unchanged.Numbers
Release build, the person tagger's 470,083-needle set:
About 180 ms / 11% off the largest single component of first-use latency. Overlapping-search results over a set of probe haystacks are byte-identical across all three kinds.
Risk
Low.
match_kindstaysStandard(required for the overlapping iteration the tagger relies on), the prefilter is untouched, and only the NFA representation changes. The full test suite passes (192 tests).