Skip to content

Commit b6ca5b5

Browse files
apcha-oaiapcha-oai
authored
ci: standardize custom-code budget reporting (#4018)
Aligns the existing custom-code budget tooling with the shared generated templates, including generated-file headers, documentation, and publisher test-fixture formatting. The existing API reference, runtime behavior, public API, and repository-owned budget policy remain unchanged. Validation: SDK formatting and all 59 offline custom-code tests passed locally, including the compiler hash contract. Co-authored-by: apcha-oai <228803254+apcha-oai@users.noreply.github.com>
1 parent 637f1b8 commit b6ca5b5

9 files changed

Lines changed: 34 additions & 19 deletions

‎.castiron.stats.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
schema_version: 1
2-
generation_id: e6e3b1ee-91de-48d6-8b7e-8021df4212ea
2+
generation_id: 63ac4032-0941-40e4-83ee-ae3cde830f2b
33
openapi_spec_hash: 8aa23d19137079c724365bd6cadd0858
44
openapi_transformed_spec_hash: d591fedadfdd68837534b2574782bf81
55
config_hash: 70e6e763ed8ac8c4038de193a96c771d
6-
codegen_sha: b56a245c7ab1a741a95e6ee6bd7357f9ca4b6e52
7-
codegen_hash: 6e391d7d5910f75be379f99a0ac4529af9a70461a6f2c7ad0a6f5d77cf00d639
8-
public_codegen_sha: 8bef4a806756c8b3db141bd6a1235fcc7148e854
6+
codegen_sha: e3fd8becca14771332e4d595da08776af0d652cb
7+
codegen_hash: 4cb3a4a438d6e8d094e5706b634fb67ebd763b682c54876704645012544862a3
8+
public_codegen_sha: bd28123c82f56c293b4b5f48f369c5f404d98f09

‎scripts/castiron/CUSTOM_CODE.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
<!-- File generated from our OpenAPI spec by Castiron. See CONTRIBUTING.md for details. -->
2+
13
# Custom code
24

35
The custom-code reporter measures the SDK's remaining customization of generated
@@ -77,8 +79,10 @@ The trusted run summary reports additions, deletions, total, mixed-file count,
7779
headroom, largest patches, and exact policy/candidate/generated revisions. The
7880
existing custom-code comment remains unchanged, including when the budget fails.
7981
The trusted compute job reuses its own report, never the candidate's artifacts.
80-
The checker, policy, and workflows are maintained in the SDK and preserved
81-
through the normal three-way merge during generation.
82+
The checker, workflows, and offline tests are generated from shared Castiron
83+
templates. The budget policy remains repository-owned and is never generated.
84+
Repository-specific customizations are preserved through the normal three-way
85+
merge during generation.
8286

8387
## Local verification
8488

‎scripts/castiron/README.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,9 @@ for cross-SDK improvements; repository-specific customizations use the normal
77
three-way merge and are allowed.
88
The reporter uses Python 3.10+, Git, and `gh`; it does not import SDK code.
99

10-
Run `python3 scripts/castiron/test_custom_code_report.py` for focused tests.
10+
Run `python3 -m unittest discover -s scripts/castiron -p 'test_custom_code*.py'`
11+
for the offline suite. Install Node.js to exercise the workflow publishers too.
12+
See [CUSTOM_CODE.md](CUSTOM_CODE.md) for budget policy and activation.
1113
The report comment includes commands to inspect the exact custom-code patch.
1214
Public reporting uses only public snapshots and needs no private repository access.
1315

@@ -17,8 +19,8 @@ Its hash format is documented in the reporter. Only `.github/actions/` and
1719

1820
The read-only pull-request workflow runs on every branch, including drafts and
1921
forks. A separate read-only `workflow_run` job computes the authoritative report from
20-
current, GitHub-associated base/head Git objects using the trusted workflow
21-
revision. It fetches those objects into a new bare repository and never checks
22+
the captured main checkout and GitHub-associated PR head. Merge groups are
23+
checked independently against current main. It fetches those objects into a new bare repository and never checks
2224
out or executes PR code. The comment-writing job consumes only the artifact
2325
from that trusted job, rechecks freshness, and links to its report and patch.
2426
PR-produced reports are advisory run output, not the published assessment. The

‎scripts/castiron/custom_code_budget.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
#!/usr/bin/env python3
2+
# File generated from our OpenAPI spec by Castiron. See CONTRIBUTING.md for details.
23
"""SDK custom-code budget gate. Run only from a trusted checkout, never PR code.
34
45
Reuses Castiron's vendored snapshot verifier and generated-file accounting. This
5-
file and its workflow are maintained in the SDK repository.
6+
file and its workflows are generated from shared Castiron templates.
67
"""
78

89
from __future__ import annotations

‎scripts/castiron/custom_code_test_support.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
# File generated from our OpenAPI spec by Castiron. See CONTRIBUTING.md for details.
12
"""Small Git/checkpoint fixture shared by the offline Castiron tests."""
23

34
from __future__ import annotations

‎scripts/castiron/fixtures/github_publisher.cjs‎

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,22 @@
1+
// File generated from our OpenAPI spec by Castiron. See CONTRIBUTING.md for details.
12
// Execute the real workflow script with an offline GitHub API and capture writes.
23
const fs = require('node:fs');
34
const data = JSON.parse(fs.readFileSync(0, 'utf8'));
45
const writes = [];
56
const github = {
67
rest: {
7-
actions: {getWorkflowRun: async () => ({data: data.run})},
8-
pulls: {get: async () => ({data: data.current}), list: 'pulls'},
9-
git: {getRef: async () => ({data: {object: {sha: data.current.base.sha}}})},
8+
actions: { getWorkflowRun: async () => ({ data: data.run }) },
9+
pulls: { get: async () => ({ data: data.current }), list: 'pulls' },
10+
git: { getRef: async () => ({ data: { object: { sha: data.current.base.sha } } }) },
1011
repos: {
11-
createCommitStatus: async value => writes.push(value),
12+
createCommitStatus: async (value) => writes.push(value),
1213
listCommitStatusesForRef: 'statuses',
1314
listPullRequestsAssociatedWithCommit: 'associations',
1415
},
1516
issues: {
1617
listComments: 'comments',
17-
createComment: async value => writes.push({operation: 'create', ...value}),
18-
updateComment: async value => writes.push({operation: 'update', ...value}),
18+
createComment: async (value) => writes.push({ operation: 'create', ...value }),
19+
updateComment: async (value) => writes.push({ operation: 'update', ...value }),
1920
},
2021
},
2122
paginate: async (method, params) => {
@@ -28,7 +29,10 @@ const github = {
2829
throw new Error(`Unexpected GitHub lookup: ${method}`);
2930
},
3031
};
31-
const AsyncFunction = Object.getPrototypeOf(async function() {}).constructor;
32-
new AsyncFunction('github', 'context', 'process', data.script)(github, data.context, {env: data.env || {}})
32+
const AsyncFunction = Object.getPrototypeOf(async function () {}).constructor;
33+
new AsyncFunction('github', 'context', 'process', data.script)(github, data.context, { env: data.env || {} })
3334
.then(() => process.stdout.write(JSON.stringify(writes)))
34-
.catch(error => { console.error(error); process.exitCode = 1; });
35+
.catch((error) => {
36+
console.error(error);
37+
process.exitCode = 1;
38+
});

‎scripts/castiron/test_custom_code_budget.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
# File generated from our OpenAPI spec by Castiron. See CONTRIBUTING.md for details.
12
# Regression tests for the custom-code budget.
23
from __future__ import annotations
34

‎scripts/castiron/test_custom_code_github.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
# File generated from our OpenAPI spec by Castiron. See CONTRIBUTING.md for details.
12
"""Trusted GitHub evaluation with real local Git objects and an offline API."""
23

34
from __future__ import annotations

‎scripts/castiron/test_custom_code_publication.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
# File generated from our OpenAPI spec by Castiron. See CONTRIBUTING.md for details.
12
"""Run the trusted publishers against offline GitHub state."""
23

34
from __future__ import annotations

0 commit comments

Comments
 (0)