baseline.md

Baseline Workflow

Agents Shipgate supports a local baseline for teams adopting the scanner after findings already exist.

Save Current Findings

agents-shipgate baseline save \
  --config shipgate.yaml \
  --out .agents-shipgate/baseline.json \
  --owner alice --reason "accepted pending Q3 fix" --expires 2026-09-30

The baseline contains active, unsuppressed findings only. Suppressed findings are intentionally excluded because they already carry an explicit review reason in shipgate.yaml.

--owner, --reason, and --expires (v0.6) record exception metadata on newly-accepted entries: who approved this debt, why, and by when it must be reviewed again. They are declared, never inferred — the same contract as human_ack (a coding agent must not manufacture approval). Entries already in the baseline keep their original metadata on re-save. To acknowledge existing unowned debt in one pass, add --apply-to-existing: it fills the given fields into entries that lack them, never overwrites a previously-set value, and preserves each entry's original recorded_at/run_id history. Use the CLI rather than hand-editing the JSON — hand-edits trip the integrity hash (SHIP-BASELINE-INTEGRITY-MISMATCH).

Severity values in the baseline are the report-visible severities after checks.severity_overrides are applied. Matching still uses fingerprints, not severity, so changing an override does not create a new baseline identity.

Debt Status And Governance Gates

agents-shipgate baseline status --json          # aging report, advisory
agents-shipgate baseline status \
  --require-owner --require-expiry --max-age-days 180   # org gate: exit 20

baseline status reports accepted-debt aging per entry (owner, reason, age, days until expiry) plus a summary (owned/unowned, expired, expiring soon, oldest entry). With no gate flags it is advisory and always exits 0; with any of --require-owner, --require-expiry, or --max-age-days N it exits 20 when entries violate the requirement — wire it into a scheduled workflow next to the host-grant drift gate to keep accepted debt owned and time-bound. An expired entry violates --require-expiry (an expired exception is no longer a valid exception), and entries without provenance (legacy 0.2–0.4 files never re-saved) fail every active gate: unknown history is ungoverned debt, not exempt debt. --as-of YYYY-MM-DD pins the reference date for reproducible CI output.

The host-grant drift gate (shipgate audit --host --drift --fail-on-drift) can exit 20 once after upgrading past 1.0.0. Claude Code hook files nothing selects now read as access: unknown, with no expansion signal, and plugin manifests, marketplaces and the hook files they select are newly read (#714). A newly read hook file carries a hook_added signal only when the repository's project settings enable the plugin that selects it, because that hook was loaded all along. A repository whose plugin manifest or marketplace declares hooks drifts once even when no grant changes: the manifest is a new artifact and a new observed source, so the gate exits 20 reporting 0 typed grant change(s) with no rows. Separately, a 0.4 or 0.5 baseline holding a workflow grant is incomparable after #771; move it aside and re-save it as the workflow step action references migration note in STABILITY.md describes. Read changes, artifact_changes and coverage_changes in shipgate audit --host --drift --json — the Markdown summary counts typed grant changes only — then move the old file aside (git mv .agents-shipgate/host-grants.json .agents-shipgate/host-grants.v0.5.json) and re-save the host-grants baseline from a checkout of the reviewed default branch. --save-baseline refuses to overwrite any baseline older than 0.6, including every baseline 1.0.0 wrote.

Apply The Baseline

agents-shipgate scan \
  --config shipgate.yaml \
  --baseline .agents-shipgate/baseline.json \
  --ci-mode strict

When a baseline is present, strict mode fails only on new unsuppressed findings that match the active fail_on policy. Existing matched findings remain visible in the report. Resolved findings are counted in the baseline summary but do not fail CI.

JSON Fields

Reports keep the v0.1 payload contract and add baseline fields:

• report_schema_version: "1.0" in current reports
• baseline.path
• baseline.matched_count
• baseline.new_count
• baseline.resolved_count
• findings[].baseline_status
• tool_surface_facts in baselines saved with schema 0.3+
• action_surface_facts in baselines saved with schema 0.4+

The public baseline comparison mode is new-findings:

agents-shipgate scan \
  --config shipgate.yaml \
  --baseline .agents-shipgate/baseline.json \
  --baseline-mode new-findings

Baseline matching uses findings[].fingerprint. The fingerprint algorithm is documented as v1: sha256(check_id | tool_name | canonical evidence)[:16], rendered as fp_<digest>. It intentionally excludes severity overrides, report paths, warnings, timestamps, default_severity audit evidence, and baseline status.

Baseline schema 0.8 additionally records support_hash for findings whose release contribution depends on typed policy evidence. The fingerprint remains stable, but both fingerprint and support hash must match. A change from heuristic, mixed, or unknown applicability to authoritative evidence is therefore reviewed as new evidence rather than silently inheriting old debt. Pre-0.8 entries have no support binding, so supported findings intentionally re-gate as new after upgrading to 0.16.0b5. Review the new evidence, then re-export accepted debt explicitly with:

agents-shipgate baseline save -c shipgate.yaml \
  --out .agents-shipgate/baseline.json \
  --owner <human> --reason "<reviewed reason>" --expires <YYYY-MM-DD>

Report schema v0.18 or higher computes public fingerprints after the default privacy redaction pass. Findings whose evidence contains a recognized secret-like value therefore get a new public fingerprint. To avoid surprise CI failures during upgrade, scan --baseline also compares the pre-v0.18 raw fingerprint in memory without writing it to public artifacts. After reviewing the v0.18 report, run agents-shipgate baseline save again to rewrite the baseline with redacted public fingerprints.

Baseline Schema Versions

agents-shipgate baseline save now writes baseline schema 0.8, which binds supported findings to support_hash. Schema 0.7 introduced provider-scoped finding identity and remains readable. Schema 0.6 adds the optional provenance.owner approver field alongside the reviewer-set reason/expires (all settable from baseline save flags). Schema 0.5 added per-entry provenance (scanner_version, run_id, recorded_at). Schema 0.4 preserves the existing finding fields, keeps the optional tool_surface_facts snapshot from schema 0.3, and adds an optional action_surface_facts snapshot so agents-shipgate scan --baseline .agents-shipgate/baseline.json can also show tool-surface and action-surface diffs when --diff-from is not supplied.

Schema 0.3 baselines still load and can drive tool_surface_diff, but they do not contain action_surface_facts; using a 0.3 baseline as the only diff reference yields action_surface_diff.enabled=false with a note. Schema 0.2 baselines still load for accepted-debt matching and strict-mode filtering, but they cannot enable either surface diff. Newer baselines are not guaranteed to load in older package versions.

--baseline and --diff-from have separate jobs: --baseline drives findings[].baseline_status, strict-mode filtering, and release_decision.baseline_delta; --diff-from drives tool_surface_diff and action_surface_diff.

In verify, --diff-from also stands in for the base scan, so it is the "before" half of every base-relative comparison, including the base policy snapshot SHIP-VERIFY-POLICY-WEAKENED reads. It is therefore never taken from the change under review (#1024). With a base (--base, or one detected), a path inside the repository is read as the base commit holds it; if the base has no regular file at that path, verify scans the base instead and says so in base_notes. A path outside the repository is the operator's and is read where it is, as is any path with --no-base. The --baseline file is read from the evaluated head. An in-repository --diff-from, --baseline or --policy-pack file that the change edits is reported by SHIP-VERIFY-TRUST-ROOT-TOUCHED (class shipgate_state, or policy for a pack), so a human reviews the edit.

Finding delta buckets compare identity, not whether the PR caused a risk. new_findings means an identity was absent from the reference; resolved_findings means it is absent from the head; unchanged_findings means the identity matched. None proves introduction, remediation or unchanged authority. accepted_debt keeps its existing reviewed-baseline meaning.

The current implementation retains full base report findings and compares their predicate support, source references, capability subjects and release contribution with matching head findings. tool_surface_diff.notes names changed evidence and base/head locations, unavailable support and ambiguous identities. The legacy buckets and gate are unchanged. A baseline or older reference without full finding evidence cannot establish that comparison; regenerate a base report to inspect it. Equal predicate support still does not establish complete shared-helper, binding or configuration dependency coverage.

The Markdown summary calls these finding identities and discloses omitted comparison notes; all notes remain in report.json. Inspect both full reports before attributing a finding to the change. #515's default diff scope remains open on dependency-proof prerequisite #557; this comparison does not exclude standing findings from the release decision.

For the OpenAI Agents SDK, the current reader also carries tool_surface_facts.guard_dependencies and compares them in tool_surface_diff.guard_comparisons. An unchanged tool declaration can now point to an imported Boolean guard whose allowed inputs changed. The report preserves the tool observation and canonical capability identity, the call and guard definition locations, the input file digests, and the base/head predicate tables. Bit i in each allowed_inputs integer represents the Boolean value of parameters[i]; the parameter list is sorted and bounded to eight names. A widened set permits more Boolean inputs, a narrowed set permits fewer, and equal sets describe equivalent predicates in that domain.

This first reader profile (sdk_boolean_guard/v1) handles a direct sibling relative import through empty package initializers, followed by a first if not guard(...): return False (or return None) in a module-level tool. The imported function has one return expression composed of Boolean parameters, constants, and, or, and not. Calls in predicates, configuration indirection, module/package ambiguity, executable initializers, rebinding, unsupported signatures, missing inputs and reader limits remain unresolved. An older report with no record does not establish absence of a guard.

These are source-predicate comparisons. They do not establish deployed Python import resolution, downstream effects, or complete capability dependency coverage. Every row reports dependency_coverage: incomplete and finding_exclusion_eligible: false; all findings and the current release decision remain unchanged. A user-supplied --diff-from report is comparison input, not authenticated proof of its claimed source evidence. Redacted or ambiguous records cannot yield a predicate direction. The Markdown report shows up to eight relevant comparisons; JSON retains the full list.

An observed guard can also carry source_behavior, from the narrower sdk_boolean_function/v1 profile. This inspects the entire selected tool function, including statements after a return and both arms of every branch, and a single source-level Agent(name="…", tools=[tool]) (or tools=[]). Its module contains only the exact SDK imports, the one sibling guard import, the decorated function and that literal Agent assignment, in definition order. The guard module contains only the selected pure Boolean function. Every function parameter is Boolean; expressions are Boolean literals, parameters, not, and, or, and the already-resolved guard call. Statements are if and return; an implicit return is recorded as none. There are no opaque operation leaves. External calls, free/configuration variables, mutation, additional functions or Agent configuration, handoffs and unresolved imports leave the source model unresolved, even in apparently unreachable code.

source_behavior.returns[i] is true, false or none for input mask i over all sorted tool parameters, not just the parameters used by the first guard. configuration_reads: none means the closed model admitted no such read; it is absent as a claim when the model is unresolved. The literal binding records the source Agent symbol/name and whether its tools list contains the selected function. This is source membership, not a claim that the Agent is a deployed entry point or that the installed SDK executes it.

The comparison keeps four axes separate: all return values (returns), the set of inputs returning true (true_domain), literal tool membership (binding), and the true-return inputs reachable through that membership (bound_true_domain). A narrowed first guard can still have a widened whole function true domain; unchanged returns can accompany added membership. Changing False to None changes returns even when the true domain is equal. A changed Agent identity leaves the bound relation unresolved. Missing old models, ambiguous capability identities and redacted evidence never establish equality. The model and its inputs are carried in each side's existing guard record and verifier-bound artifacts; it introduces no separate authority.

A true return is not approval. The model does not prove a caller-controlled Boolean came from an approver, establish an action's effect/authority or attribute an existing policy predicate to a change. The outer capability dependency coverage remains incomplete and finding exclusion remains false. Neither a narrowed true domain nor equal source models is a safe verdict. Joining complete source evidence to actual capability/policy dependencies and the fixed-history evaluation remains work in #557/#515/#563. This SDK profile does not cover #515's TypeScript MongoDB acceptance case.

Verification binds the reader's captured dependency bytes and named absent import candidates in verification-plan.json under inputs.options.dependency_inputs. A current-control read checks them again, including ignored files and newly appearing packages that ordinary Git status would miss. Committed comparisons read each side's archived Git tree. This input binding does not turn a bounded predicate into a complete dependency closure; #557 and #515 retain that remaining obligation.

An attempted guard read that fails before bytes can be captured (such as a symlink, oversized input or non-regular file) is explicitly unconfirmable. agent control then uses its existing workspace_unverifiable refusal; repair the unreadable path and re-run verification before using current authority. This does not replace the report's release decision. A missing candidate whose absence was captured, or an unsupported predicate whose bytes were read, remains confirmable. Durable typed refusal identities are deferred separately; an exception string is not sufficient evidence of stable inputs.

Scope deltas distinguish tool-required scopes from manifest-declared scopes. If the same literal scope moves between those kinds, the diff reports one removed scope and one added scope so the JSON preserves the source of the change.