bench: rebuild suite into zero-dep guard tier + cross-framework comparative tier#332
Merged
Conversation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e its pytest config
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…benchmark doc refs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds the benchmark suite around two tiers sharing one scenario vocabulary. Full rationale and design in the change file:
planning/changes/2026-07-15.05-benchmark-suite-rebuild.md.What changed
benchmarks/, zero external deps) — 7 scenarios (G1–G7) decomposing modern-di's resolve hot path: transient, cached, depth-6 chain, wide fan-out, cross-scope, child-container build, and the full request lifecycle with a sync-init/async-finalizer teardown. Replaces the four stale fix#1–4 micro-benchmarks (which compared code to monkey-patched reimplementations) andRESULTS.md.benchmarks/comparative/, isolated standalone uv project) — C1–C4 for modern-di + dishka 1.10.1 + that-depends 4.0.2 + dependency-injector 4.49.1 + wireup 2.12.0, same graph shape, each in its own idiomatic setup. Local-only viajust bench-compare; competitor deps live only in the nestedpyproject.toml, so the root stays zero-runtime-dependency..github/workflows/benchmarks.yml) — runs the guard tier on push/PR and posts a non-gating PR comment with the per-scenario delta vs a rollingmainbaseline (cached viaactions/cache, no gh-pages). Never fails the build.benchmarks/README.md(living methodology + honest per-framework mapping),just bench(guard) /just bench-compare(comparative).Notes for review
pyproject.tomlgains a lint-only exclude ofbenchmarks/comparative(ruff +[tool.ty.src]) so the root toolchain doesn't try to type-check competitor imports — no[project]dependency added; and the CI baseline uses a unique-key +restore-keyscache pattern (Actions cache entries are immutable, so a static key would freeze the baseline).Verification
just test-ci: 392 passed, 100% coverage (no library code changed).just bench: 7 passed.just bench-compare: 20 passed (5 frameworks × C1–C4), each asserting the resolved graph (C4/G7 assert the async finalizer ran).just lint-ciandjust check-planning: pass.🤖 Generated with Claude Code