Sync reporting to the dynamic report — 3 fixed tools retired, metamodel pair in [DO NOT MERGE until prod release] - #19
Draft
amitl-levi wants to merge 3 commits into
Draft
amitl-levi wants to merge 3 commits into
amitl-levi wants to merge 3 commits into
Conversation
…metamodel pair in Upstream replaces get_top_campaign_content_report, get_campaign_breakdown_report, and get_campaign_site_day_breakdown_report with get_dynamic_report_settings + get_dynamic_report_data (any dimension x metric combination; settings-first is mandatory - the metamodel is the only source of valid field names). get_campaign_history_report survives as the change/audit log, reframed per upstream: it is not performance data. The banner drops the grand Total (now Records | Grain | pagination), so the pagination stop rule becomes page-until-short-page and skills state fetched scope instead of citing Total. reporting-aggregation.md is rebuilt around that and gains the sum-vs-ratio / weighted-re-derivation rules; server-side aggregation (ask for the grain you want) is now the default posture. Staging-validated 2026-08-20 (10 questions, raw counters exact); the quirks found there ride along marked staging-observed, re-verify at release: CTR uses visible impressions, Week starts Sunday, SITE.DESCRIPTION over SITE.NAME, entity-attribute dimensions may return unaggregated rows. Reporting account types documented (PARTNER/NETWORK yes; GROUP/admin network 403 by design). Every stale claim site updated per the stale-capability-claims policy: agent tool reference + examples + boundary, reports skill + both references, optimize-campaign dimension table, campaigns flow, diagnose-tracking spend check, CLAUDE.md diagram + CSV section + new design note, README, read scenarios 5-9. The prepared comparison questions and the reports-gaps audit (with a migration status update) are committed alongside. Version to 0.5.0 (breaking: three tools leave the surface). DO NOT MERGE until the dynamic report tools are live on production mcp.realize.com - the PR carries the release gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two fresh-context reviewers audited the sync commit — one against the upstream tool contract, one for internal consistency. 26 findings, the real ones fixed here: - optimize-campaign/SKILL.md was missed by the sync entirely (the one skill file the sweep's tool-name grep couldn't catch, because its stale claims described the history report without naming a retired tool): P1 tracking health extracted conversions/clicks from what is now an audit log -> routed through a campaign-grain dynamic report; RCA Signal 1 asserted the history report returns performance time-series, not change events - the exact inverse of the new world -> it now pulls the changelog and keeps ask-the-user as fallback. - Agent Core Responsibilities #4/#5 contradicted the same file's own updated spec (Total-bearing banner example, covered-the-Total stop rule) -> Grain-form example + short-page stop rule. - Tool count 19 read -> 18 in four sites (agent x2, CLAUDE.md diagram, best-practices-gap baseline). - optimization-flow.md pointed at 'the mandatory Total read' in a knowledge file this same PR rebuilt without one. - Read scenarios: 11 expected Total: 0 from a bannerless-Total tool; 9a routed a performance check to the audit log; 9 required UI paths for prescriptions the skill routes through manage-campaigns; 6 now includes the status column and a disclosed post-filter fallback if the metamodel lacks a status filter field. - Hedging: Week-starts-Sunday and visible-impressions CTR now carry the staging-observed tag CLAUDE.md claims all quirks carry; the dimension/metric enumerations are attributed to the Realize UI list with the metamodel named authoritative; the dynamic test doc's 2-dimension cap is labeled a UI-parity protocol constraint, not a tool limit. - tests/README truncation advice keyed to the TRUNCATED banner; manifest keywords retagged (Grain header / short page / dynamic report). Findings rejected with evidence: the CTR ranges and 400 error string are verbatim from the staging findings; name-or-label matching and the markdown-menu format are upstream's own wording; search_accounts Total is stated in its upstream description. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two more independent reviewers: one walked six real user questions through the docs as the runtime model would; one re-reviewed the cumulative PR (both commits), the PR body, and structure. 21 findings; the real ones fixed here. Most are pre-existing bugs the simulation surfaced - they predate this PR but broke loudest under the new tool surface: - Sum-reconciliation gate looped forever on bounded windows: get_campaign.spent is lifetime, not window-scoped, so any July-style historical pull failed the 2% gate by construction. Reference is now a same-window campaign-grain report. - optimize-campaign mandated checks no tool can run: auction insights (now: UI-sourced, ask the user, never fabricate a loss %), RCA Signal 4 fired/matched ratios (now: attributed-conversions trend + diagnose-tracking handoff), frequency and SpendGuard state (now: ask the user), Signal 6 change timestamps (now: change log where visible, else ask). - P3 (active conversions only) got an executable recipe: rule statuses via get_conversion_rules x per-rule metrics; the aggregate metric can't exclude dead rules server-side. - Action prescriptions routed MCP-writable fixes to UI paths - the exact stale-capability failure the agent forbids. All seven bullets now route through manage-campaigns. - The daily-spend >= 8x CPA goal floor was promised by the agent, README, the capability baseline, and two test scenarios but missing from the skill after an earlier rewrite. Reinstated in the data-sufficiency gates with its toolkit provenance (Apr 2026). - Re-derived CTR vs report CTR: the mandated weighted re-derivation uses total impressions, the report uses visible impressions - now stated so nobody ships the two in one comparison. - History report: legacy page/page_size pair restored to its signature; account-wide scope + client-side campaign filtering documented; week-grain queries now snap to Sun-Sat whole weeks. - Reports/optimize routing boundary drawn (trend explanation vs what-do-I-do); agent example sizes now respect the page cap; stale start_date/end_date advice fixed; CLAUDE.md no longer lists the missing grand Total as a deletable staging quirk; scenario 6 gained a list_campaigns join fallback; attribution-basis gap flagged in the reports skill; changelog claim-site list completed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
🚫 DO NOT MERGE until the dynamic report tools are live on production
mcp.realize.com. Upstream's change (Bitbucket: DEV/realize-mcp-server PR #13) is still open, and the tools were validated on staging only. Merge gate before flipping this to ready: (1) livetools/liston prod showsget_dynamic_report_settings/get_dynamic_report_dataand drops the three retired tools; (2) re-check the shipped schema against the skill docs (the upstream PR is still moving); (3) re-run 2–3 of the 10 prepared comparison questions against prod; (4) re-verify each staging-observed quirk and delete the ones that got fixed.Part 6 of the stack. Syncs the plugin to upstream's dynamic-report release: the metamodel pair
get_dynamic_report_settings+get_dynamic_report_datareplaces the three fixed-grain report tools (get_top_campaign_content_report,get_campaign_breakdown_report,get_campaign_site_day_breakdown_report);get_campaign_history_reportsurvives, reframed per upstream as the campaign change/audit log, not performance data.Stack: originally based on
diagnose-tracking(#16, now merged); base is nowmain; this PR's diff is its three commits: the sync plus two review-fix commits. (Reopened as a new PR: the original #17 was auto-closed when its stacked base branch was deleted on #16's merge.) Merge order: #13 → #14 → #16 → this.What
reportsskill rewritten around the mandatory two-step workflow (settings first — the metamodel is the only source of valid fully-qualified field names; guessing is the tool's documented failure mode), the query rules (date_presetXOR custom range, structured filters with operators, sort-column-must-be-selected, top-N pattern), and the new banner (Records | Grain | pagination— no grandTotal, so the pagination stop rule becomes page-until-short-page and summaries state fetched scope).knowledge/reporting-aggregation.mdrebuilt: server-side aggregation as the default posture (request the grain you want), sum-vs-ratio column rules with weighted re-derivation (closing a documented gap indocs/reports-gaps.md), and the sum-reconciliation gate adapted to the no-Totalworld.SITE.DESCRIPTIONoverSITE.NAME, entity-attribute dimensions returning unaggregated rows.optimize-campaigndimension table,campaignstop-campaign flow,diagnose-trackingspend check,CLAUDE.md(diagram, CSV section, new design-decision note),README, read Scenarios 5–9.tests/test-scenarios-dynamic-report.md(the 10 prepared comparison questions + run cards) anddocs/reports-gaps.md(the reports audit, with a migration status update).Why held
Production still serves only the 4 fixed report tools. Shipping these docs early would make the plugin fabricate calls to tools that don't exist yet — the exact stale-capability-claims failure the sync policy exists to prevent, in the opposite direction.
🤖 Generated with Claude Code
Review rounds (4 independent reviewers + 2 self-review passes)
Commit 2 (
b636141) and commit 3 (4ced4de) carry the fixes from four independent fresh-context reviews: a schema audit against the upstream tool contract, a repo-wide consistency sweep, a usage simulation (six real user questions walked through the docs as the runtime model would), and a cumulative structural review. 34 verified findings fixed across the two commits — the biggest:optimize-campaignhad the history report's meaning inverted and mandated checks no tool can run (auction insights, fired/matched ratios — now UI-sourced/ask-the-user), the sum-reconciliation gate failed by construction on bounded date windows, and the daily-spend ≥ 8× CPA goal floor promised in four docs was missing from the skill (reinstated). Full detail in the two commit messages.