Skip to content

Commit cf8150b

Browse files
feat(vex): manifest-less VEX from hosted/vendored lockfiles + npm 12 allow-remote auto-config (v5) (#251)
* fix(core): hosted/vendored rewriter fixes found by the real-PM matrices First of seven commits that land the manifest-less VEX branch (feat/vex-lockfile-inventory) on top of #247. The branch's ~165-commit history, built on the pre-#247 main 9489b18 with per-PM merge commits, was squashed onto 09956d9 and re-split by concern; the full pre-rebase history is preserved on branch backup/vex-lockfile-pre-rebase. These are the product bugs the per-PM real-toolchain matrices turned up while the manifest-less VEX suites were written. Each is independent of VEX and pinned by a core regression test: - composer (hosted): the rewriter drops the entry's `source` block when it immediately precedes `dist` (one `redirect_composer_dist` fragment edit spanning both, reverted byte-for-byte). Composer 1 and 2.2 LTS silently installed the pristine upstream commit from git whenever the hosted download failed; a hand-ordered source that cannot be dropped warns `redirect_composer_source_kept`. Golden fixture: `source-and-dist`. - gem (hosted): the patch-registry `GEM` section is inserted in bundler's source order (sorted by remote), so a frozen `bundle install` on bundler >= 4.0.19 accepts the converged lock. The `basic` golden and the exact lock expectations move the Socket section first. - cargo (hosted + vendored): a v1 `Cargo.lock` (checksums in `[metadata]`, dependents naming the crate by its full package id) is redirected and vendored correctly: the `[metadata]` line and every dependent's full-id reference follow the source, each fragment its own ledger edit, and `cargo --locked` accepts the result; the vendored detach/restore of a v1 entry is byte-identical. `plan_cargo_lock` keeps #247's multi-source twin disambiguation (`Ambiguous`) and hoisted regexes; the block end now also stops at a trailing `[metadata]` / `[[patch.unused]]`, and the checksum is inserted after a block-final `source` line too. - yarn berry (hosted + vendored): written checksums follow the lock's own spelling — yarn 4.0.0–4.0.2 spell `10c0` checksums as bare hex, so the prefixed form failed `yarn install --immutable` with YN0028. - npm (vendored): npm 12 reifies from the `package-lock.json` it creates beside a committed shrinkwrap, so `vendor` now rewires every present npm lock (siblings first, primary last; an unrewirable sibling warns `vendor_npm_sibling_lock_unwired`), and the in-use/revert probes read every npm lock before deleting an artifact. `select_lockfile` reads through #247's guarded `read_regular_to_bytes`. - npm (hosted): a lockfileVersion 1 redirect warns `redirect_npm_legacy_client` — npm 6 ignores a v1 lock's `resolved` and fails EINTEGRITY against the patched pin. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(hosted): auto-configure npm 12 allow-remote in the project .npmrc npm >= 12 defaults to `allow-remote=none` and refuses (EALLOWREMOTE) every lockfile entry whose tarball is not served by the configured registry — exactly what a hosted redirect writes. When `scan --mode hosted` / `get --mode hosted` leaves a root package-lock.json / npm-shrinkwrap.json carrying a granted hosted artifact URL, the run now ensures `allow-remote=all` in the project `.npmrc` (creating the file, or appending one line with the BOM, CRLF and every other byte preserved) and records it in the redirect ledger as `redirect_npmrc_allow_remote` (`created` / `added`). - core `patch::redirect::npmrc`: npm's `.npmrc` grammar as measured against npm 12.1.0 (exact `allow-remote` key, last top-level assignment wins, `[section]` bodies are not top-level, bare-CR line splits, case- sensitive value), the plan (create / append / already-all / respected user / env / outer-layer value / unsupported), and the unwinds. - Every reversal removes exactly what was added once no package-lock entry needs it: the whole-ledger replay (a new `NpmrcAllowRemote` inverse, grouped with the npm lock kinds), the per-purl npm revert behind scoped rollback / remove / the vendored takeover ("last one out", same transaction, flushed after the lock through #247's shared `staged::flush_staged`), and the vendored-supersedes-hosted reconcile. A modified created file keeps the user's lines (`redirect_npmrc_allow_remote_modified`, surfaced by rollback, remove, vendor and the reconcile). A symlinked `.npmrc` refuses an unwind at plan time, before anything is written. - Hosted run: the `.npmrc` edit rides `rewrite.files` / `rewrite.edits`, so it is written under #247's apply-lock window, after its whole-run SYMLINK GUARD, and only after the ledger persisted — never on `--dry-run` (which previews the write, also for a vendored → hosted takeover; the root locks are now read for such a preview so the pnpm `trustLockfile` preview sees the lock the wet run splices). An explicit user value (project `.npmrc`, user/global/builtin config, or an `npm_config_allow_remote` env var) is respected and named; a symlinked, unreadable or bare-CR `.npmrc` is left alone; `--no-npm-allow-remote- config` / `SOCKET_NO_NPM_ALLOW_REMOTE_CONFIG` opts out. Every variant warns `redirect_npm_allow_remote` with the whole-tree tradeoff. - `atomic_write_bytes_preserving_mode` creates its stage with the destination's permission bits (kept inside #247's `commit_stage` structure), so a 0600 token-bearing `.npmrc` is never staged world- readable. - remove: the hosted leg's advisories are printed inside #247's `unwind_hosted` (so a run that then fails still reports them) and carried into the success envelope's `warnings[]`. Tests: npmrc/replay/takeover/scan unit tests, the flag's parse coverage, `redirect_npm_allow_remote` (plus #247's invariants: the lock never outlives the run, dry runs create no `.socket/`, a full rollback leaves no `.socket/` and never the user's `.npmrc`), and the dry-run takeover previews in `coverage_fix_scan_hosted_dryrun_vendored`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor(core/vendor): split lock_inventory into per-format submodules A verbatim move with no behavior change, so the next commit can give each lock format its shared entry model beside its registry view: `vendor/lock_inventory.rs` becomes a directory module with one file per format (`npm`, `npm_family`, `pnpm`, `yarn`, `bun`, `cargo`, `golang`, `composer`, `gem`, `pypi`), ledger recovery (`recover`) and the rewired-lock trust anchor (`wired`). `mod.rs` keeps the public API (`LockIntegrity`, `LockfileEntry`, `UnsupportedNpmLayout`, `lookup`, `inventory_project(_diagnosed)`, `recover_lock_entry`, `wired_vendor_integrity`); the three test modules move to `tests.rs`, `recover_tests.rs` and `python_lock_union_tests.rs`. Only imports, visibility (helpers another file calls become `pub(super)`), sibling-module paths (`super::state` -> `crate::vendor::state`), module docs and the single-file section banners changed; the test modules lost one indentation level and were re-wrapped by rustfmt. `git show --color-moved` shows everything else as moved. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(core/vex): discover hosted and vendored patch references from lockfiles New read-only module `vex::discover`: `discover_patched_refs(_with)` reads every supported ROOT lockfile / package-manager config and returns the Socket patch wiring it finds — `Discovery { refs, diagnostics, recognized, unlocked_pins, elsewhere }`, one `PatchedRef { purl, uuid, mode (Hosted | Vendored), source_file, artifact_rel, locked_integrity, integrity_required, url }` per live reference. It never touches the network, never writes, and never fails the run: a malformed file is a diagnostic (`lockfile_unreadable`, `lockfile_unparseable`, `patched_ref_invalid`, `patched_ref_unattributable`). One extractor per package-manager family: npm (package-lock / shrinkwrap, pnpm every lock generation + Rush locks), yarn classic and berry, bun (`bun.lock` / `bun.lockb`), cargo, golang (go.mod / go.work + sums), pypi locks (uv, PEP 723 script locks, pylock, poetry, pdm), pypi other (Pipfile.lock, requirements + `-r`, Hatch / PEP 621 direct refs), gem, composer, maven, nuget; deno is explicitly empty. Every file present is read — no precedence chain — because the hosted rewriter edits every candidate it finds. Every value is committed, tamperable data and is validated fail-closed: - hosted: `hosted_patch_uuid` accepts only `https://patch.socket.dev` or a configured `--patch-server-url` origin, no userinfo, and takes the LAST canonical-uuid path segment (grant tokens may be uuid-shaped); percent-encoding, `\/` escapes and fragments are handled; - vendored: root-anchored `.socket/vendor/<eco>/<uuid>/…` paths whose leaf names the entry's own artifact (`vendor_ref` takes the path literally; only yarn / URL-form pip strip their `#…` / `::…` decorations); - pins, not definitions: a registry / index / source definition alone (cargo `[registries]`, nuget `<add>`, pom `<repository>`, uv index tables, `.npmrc`) never makes a reference; - contested locks: a lock that resolves the same name@version from a non-Socket source drops the ref (`patched_ref_unattributable`); - lockless cargo pins / exclusive nuget mappings are recorded as `UnlockedPin`s that can only keep a ledger record live, never create a ref; `recognized` lists every uuid a read file mentions, so a rejected mention keeps nothing alive downstream. Supporting core changes: `patch::redirect::{SOCKET_PATCH_SERVER_HOST, hosted_patch_uuid, hosted_patch_url_uuids}` (the pipenv owner check uses the shared host constant); the pnpm lock grammar and `hosted_url_version` exported crate-wide so readers parse exactly what the writers write; `utils::digest` (the SRI pin rule and the hex digest shapes — one copy for the inventory, discovery, ledger recovery and the rewriters, each call site keeping its case policy); and `utils::purl`'s validating purl builders, which discovery and the inventory's registry views share. A few writer helpers become `pub(crate)` so the extractors' tests derive their fixtures from the writers themselves. One lockfile traversal layer: discovery and `vendor::lock_inventory` (the scan / get / vendor / repair inventory) read each format through ONE reader that yields every entry, Socket-owned ones included — the inventory's registry views drop those, the extractors classify them: - `lock_inventory` becomes a directory module, one file per format, each laid out as a pure entry model, a stat-only file-selection section and the registry view (an architecture test enforces the layering). Entry models: `npm_lock_nodes`; `pnpm_packages` over the hosted rewriter's pnpm grammar (every key generation, CRLF included); the yarn `classic_entries` / `berry_entries` models with one berry locator, cache-key and checksum rule; `BunLockb::parse_packages`; and `composer_lock_packages`, whose array index the composer writer's lock walks use too. - Every other format reads through the reader its writer owns: `cargo_lock::locked_packages` and `cargo_config`'s `[patch]` / `[registries]` walks; `go_mod_edit` / `go_sum_edit` read helpers; a new `vendor::gemfile_lock` model; the `utils::python_lock` / `poetry_lock` / `hatch` readers (uv source fields, script-lock pairing, the pyproject / Hatch declaration walk); a new `utils::requirements` lexer lifted out of the vendored requirements planner; and two new XML readers, `vendor::maven_pom` and `vendor::nuget_config`, with `nuget_feed::nuget_lock_entries` shared by discovery and the feed writer. - `DiscoverCtx::locate` classifies a lock location once (vendored path, hosted uuid, decorated leaf) for every extractor. The npm-family extractors iterate the entry models only, never the grammar primitives, and every extractor reads content only through the recognizing ctx (rule 11) — both enforced by architecture tests. Ledger liveness is one rule too: `Discovery::{wires_package, vendor_entry_live, redirect_record_live}`, held per call site by `LedgerLiveness` (the sorted redirect-ledger files, the lock inventory loaded lazily at most once), which the CLI's vex and scan both use. Cargo crates.io provenance is explicit (`LockfileEntry::source_kind`), not inferred from the checksum variant. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(vex): manifest-less VEX — attest hosted and vendored patches from lockfiles `socket-patch vex` and the embedded `apply` / `scan` / `vendor --vex` now attest hosted and vendored patches with no `.socket/manifest.json` — and with no `.socket/vendor` ledgers either — by reading the wiring out of the project's lockfiles (`vex::discover`). This covers a depscan-opened PR, a clone that never committed its ledgers, and a lock-only CI checkout. Record view (`commands::vex_sources::plan`): four sources — the manifest, the redirect ledger's records, the vendor ledger's embedded records and lockfile discovery — merge into one view. A candidate's record must carry the uuid the lockfile actually WIRES; it comes from the first source that has one, else (online only) from the patch API by uuid, 10 fetches at a time with `get`'s one-shot 401/403 → public-proxy fallback (#247's uuid-only `ApiClient::fetch_patch`). Nothing is written: vex never takes the apply lock, never creates `.socket/`, never writes the manifest, and the `socket-patch.vendor.json` marker is never a record. Gates, applied before hashing and kept under `--no-verify` (which now skips only the hashing): - a vendor-ledger entry attests only while a lockfile still wires its artifact (`vendor_unwired`); a redirect-ledger record only while a lockfile wires its hosted patch (`redirect_unwired`); discovery is authoritative for every uuid a read file mentions, so a rejected mention keeps nothing alive; - `record_unavailable` (offline, 404, refused, transport error — the run continues), `record_mismatch`, `wiring_conflict` (lockfiles wire one package to several patches); - a malformed / unreadable `.socket/vendor/state.json` is the hard error `vendor_ledger_corrupt` (exit 2), mirroring `redirect_ledger_corrupt` — this supersedes #247's degrade-and-disclose posture for `vex` only; `setup --check` keeps #247's `vendor_context_from` / `warn_unreadable_vendor_state` path. Evidence: vendored refs hash the committed artifact (the ledger entry when it names the wired artifact, else one synthesized from the ref); hosted refs hash the installed copies the build CONSUMES (`vex_consumed::hosted_consumed_copies` → core `VendorContext::hosted` / `HostedCopies`: the Go replacement module, the Socket-registry cargo src dir, maven's suffixed version — never a pristine sibling), and with nothing installed a discovered pinned reference attests from its lockfile pin, like in-run `scan --mode hosted --vex`. Discovered refs bypass the Property 7 ecosystem filter. Commands: - `apply --vex` / `vendor --vex` with no manifest attest what the lockfiles and ledgers wire (nothing anywhere keeps the calm exit 0 and removes a stale document; `apply --check` and `--dry-run` never generate); #247's no-manifest lines ("No patch manifest found; nothing to apply.", "No manifest found, nothing to vendor.") are kept; - failed VEX runs carry the discovery diagnostics into `warnings[]` (standalone envelope, embedded envelopes, scan JSON — hosted included); - `manifest_not_found` now means no manifest AND nothing wired; - human output: `Note:` lines for superseded records / fetch failures, phrased omission reasons. Writer hardening: the manifest-driven standalone `vendor` now embeds the patch `record` in its ledger entries too (vendored mode already does, as `detached` entries). `detached` stays the "no manifest owner" flag, so the manifest reconcile, legacy-manifest migration and get/scan idempotency from #247 are unaffected. Every reader of embedded records shares one ownership rule (`commands::vendor_record_is_unowned`): a detached entry's record always stands alone, a standalone `vendor` entry's fallback copy only when no manifest entry covers it (by ledger key or base purl). `vex`, `list` and `setup --check` (`fold_vendor_records`, formerly `fold_detached_records`) all apply it, so one tree never lists "no patches" while its VEX document attests one. `repair` stays narrower: it keeps preferring a manifest that moved on to a newer uuid and falls back to the embedded copy only with no manifest at all. One liveness rule for vex and scan: `scan`'s cross-mode takeover classification (`classify_overlap_takeover`), its `hosted_wiring_retained` warning and `redirectState.wiringLive` ask the same core discovery (`commands::discover_wiring`) and liveness rule (`Discovery::redirect_record_live` / `vendor_entry_live` / `wires_package`, through one `LedgerLiveness` holder per call site) that gate attestation, replacing scan's private cargo / hosted / vendored checks and its looser text scan. The CLI also shares one purl splitter (`utils::purl::purl_parts`), one vendor-ledger lookup (`vendor::state::lookup_entry_kv`) and one npm alias-aware identity crawl (`ecosystem_dispatch::npm_paths_by_identity`) across vex, scan and vendor, and pairs PEP 723 script locks through `utils::python_lock::script_of_lock` like the rewriters do. Output follows the conventions from #248: `ui::plural` counts, a `ui::StatusLine` progress line for record fetches, and the shared `format_vex_written` / `format_vex_dry_run_skip` lines on the manifest-less `apply --vex` / `vendor --vex` paths. Manifest-less `vex` honours `--dry-run` and `-O -` like the manifest path. Product auto-detection adds go.mod, composer.json, pom.xml, a single `*.csproj` and a single `*.gemspec`, after the existing probes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(cli): manifest-less VEX suites, real-PM capstones and shared helpers Hermetic suites (wiremock patch API, run in the default `test` job on every OS): one `e2e_vex_lockfile_<pm>` per package manager — npm, pnpm, yarn (classic + berry), bun, cargo, golang, uv, poetry, pdm, hatch, pipenv, pip, gem, composer, maven, nuget, deno (negative) — covering spoofed hosts, record mismatch, pristine / tampered installs, contested and orphaned wiring, `--offline`, reverted locks (also `--no-verify`) and the embedded commands; plus the cross-cutting `e2e_vex_manifestless_ embedded`, `e2e_vex_redirect`, `e2e_vex_vendor` and the `vex_e2e_common_selftest` harness checks. Real-PM capstones (gated like the existing suites, `_REQUIRED` env pattern): every hosted / vendored build suite now ends in the manifest- less VEX matrix over a fresh checkout — manifest absent (ledgers kept), ledgers deleted (lockfile + API), offline (`record_unavailable`, zero requests), reverted lock (never attested, verify or not) — through shared helpers: `vex_e2e_common` (+ bun / uv matrices), `npm_e2e_common` (+ `manifestless`), `yarn_berry_common`, `common/yarn_classic_vex`, `common/bundler_e2e`, `cargo_e2e_matrix`, `golang_e2e_matrix`, `maven_build_common`, `composer_e2e_common`, `vex_pdm_hatch_common`, `vex_pipenv_pip_*`, `vex_pypi_real_common`. New real-toolchain suites: `e2e_redirect_{composer,maven,uv}_build`, `e2e_vendor_maven_build`, `e2e_golang_workspace_build`, `e2e_nuget_dotnet_build`, `e2e_poetry_vex_build`, `e2e_vex_{pdm,pip,pipenv,hatch}_build`, `e2e_deno_vex_build`; the docker / setup-matrix / production legs gain their manifest-less VEX steps. Re-pinned to #247 while rebasing: - vendored mode writes no manifest: the "manifest deleted" steps are naturally manifest-free, so the npm manifest-less matrix and the uv / poetry vendored suites add a `legacy-manifest` cell (the record a pre-5.0 vendored run left beside its ledger, via `vex_e2e_common::seed_legacy_manifest`, must attest the same way), the yarn 2/3 refusal suite seeds that legacy record explicitly (and still requires `not_applied`, never an attestation), and `--detached` twins assert no manifest in either spelling; - a fully reverted project keeps no `.socket/`: suites that plant a stale ledger back after a rollback recreate the directory first, and the hosted rollback suite checks that `vex` does not recreate it; - the no-manifest human lines follow #247's wording; - a corrupt vendor ledger on a manifest-free project is now `vendor_ledger_corrupt` (exit 2, the ledger named, never rewritten), replacing #247's disclose-then-`manifest_not_found` expectation; - repair: two tests pin the standalone-`vendor` embedded record (a moved-on manifest still wins; with no manifest the embedded copy stands in offline), replacing the pre-rebase test that assumed a non-detached `scan --vendor` entry. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci: run the manifest-less VEX real-PM matrices and backtest steps ci.yml: - `test` exports `SOCKET_PATCH_GO_E2E_REQUIRED` / `_VERSION` so the real-go hosted/vendored suites fail instead of skipping on the Go that is already installed for vexctl; - `e2e` gains pinned-toolchain legs, each ending in the manifest-less VEX matrix: composer 1 / 2.2 / 2 (hosted + vendored), bundler 1.17.3 → 4.0.21 (hosted + vendored + setup_matrix_gem), bun text/binary-lock eras, the named corepack pnpm legs (3 OS), uv 0.1.45 → 0.12.x hosted + vendored, poetry / pdm / hatch / pipenv / pip, maven 3.6.3 → 4.0.0-rc-6, dotnet 6 → 10, deno; npm legs hard-require npm where `Command::new("npm")` can resolve it; - new jobs: `yarn-classic-matrix` (1.0.2 → 1.22.22), `yarn-berry-e2e` (4.0.2 → 4.18.0 + macOS/Windows), `cargo-vex-matrix` (toolchains × Cargo.lock v1–v4); `e2e-docker` also runs the pypi vendored-PM suite. Compatibility workflows: npm (npm 6–12 capstones, new), go (1.18.10 → 1.26.3, new), poetry (new), and the pnpm / pdm / pipenv / bun ones run the manifest-less VEX steps and trigger on the vex sources. The bun workflow keeps #247's two modes (hosted, vendored — the vendored- detached leg collapsed into vendored). Scripts: the bun / pdm / pipenv / poetry / uv backtests gain the manifest-less VEX checks (a fresh checkout of the committed state must be attested, also with the ledgers deleted; `--offline` with no ledger omits `record_unavailable`; a reverted lock never attests, also under `--no-verify`; a refused lock format attests nothing — the pdm refusal check follows #247's manifest-free footprint), plus the `{uv,yarn-berry,yarn-classic}-vex-matrix.sh` drivers and harness unit tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs: manifest-less VEX, npm 12 allow-remote and the v5 changelog - CHANGELOG [Unreleased] (both entries are v5 MAJOR): the semver note now lists #247's changes and `vex` refusing stale ledger records and corrupt vendor ledgers; Changed (BREAKING) gains the ledger-liveness / `vendor_ledger_corrupt` entry (and #247's Fixed bullet that had `vex` disclose an unreadable ledger now points at it); Added gains the npm 12 `allow-remote` auto-config, manifest-less VEX, standalone `vendor` record embedding and the new product probes; Fixed gains the composer / gem / cargo v1 / yarn 4.0 / npm dual-lock rewriter fixes after #247's own entries. - CLI_CONTRACT.md: new "Manifest-less VEX (lockfile discovery)" section (inputs, per-ecosystem recognition table, record resolution, verification basis, liveness gates, run warnings), "Patch hosts", the embedded-VEX no-manifest rules, `manifest_not_found` for `vex`, the vendor ledger's `record` semantics (vendored-mode `detached` entries plus the standalone `vendor` fallback copy; the reconcile exemption keys on `detached`), and the new rollback warning code. - README: "No manifest needed for hosted and vendored patches", the rewritten `vex` how-it-works steps and product probes, "npm compatibility (hosted mode and npm 12)", the hosted `.npmrc` commit hint, and the ledger-liveness note under "Undo things". - docs/testing: npm-compatibility.md (npm 6–12, new), uv and bun tables, ecosystems.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(vex): golden snapshot of discovery over committed fixtures Pin the exact output of lockfile discovery over every committed fixture project, so a behavior change to discovery (or to the lock readers it shares with the rewriters and vendor::lock_inventory) shows up as a golden diff to review. - Corpus: 182 projects under crates/socket-patch-core/tests/fixtures/ — every redirect case's input/ and expected/ tree, pnpm-hosted, poetry, pipenv, bun-lockb captures, and each pdm-native lock staged as pdm.lock — run through the full orchestrator and checked against rule 11's recognition-covers-refs invariant. - Rendering (src/vex/discover/testing/golden.rs): every PatchedRef field (plus lockfile_basis_ok), diagnostics (tempdir root and OS error numbers normalized), recognized identities, unlocked pins, resolved-elsewhere entries, and the live hosted / vendored ledger claims. The destructuring is exhaustive, so a new field fails to compile until the golden covers it. - Goldens: one JSON per fixture family (redirect-<eco>, bun-lockb, pdm-native, pipenv, pnpm-hosted, poetry) under tests/fixtures/vex-discover-golden/, mapping fixture path to output. Missing and orphaned entries fail. Regenerate after an intended change with SOCKET_PATCH_UPDATE_GOLDEN=1 cargo test -p socket-patch-core --lib vex::discover::testing::golden. Unix only: Windows checkouts convert some fixtures' line endings. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent e024629 commit cf8150b

275 files changed

Lines changed: 94244 additions & 8282 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitattributes‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,8 @@
55
crates/socket-patch-core/tests/fixtures/redirect/** -text
66

77
crates/socket-patch-core/tests/fixtures/pdm-native/*.lock -text
8+
9+
# The captured pnpm 1-12 locks are byte-real: the hosted/vendored rewriters
10+
# refuse CRLF by design (vendor_lockfile_crlf_unsupported), and the tests
11+
# derive their CRLF variants from the LF bytes themselves.
12+
crates/socket-patch-core/tests/fixtures/pnpm-hosted/** -text

‎.github/workflows/bun-compatibility.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,12 @@ on:
3434
- 'crates/socket-patch-core/src/utils/process.rs'
3535
- 'crates/socket-patch-core/tests/fixtures/redirect/npm/bun/**'
3636
- 'crates/socket-patch-cli/src/commands/get.rs'
37+
- 'crates/socket-patch-core/src/vex/**'
38+
- 'crates/socket-patch-cli/src/commands/vex.rs'
39+
- 'crates/socket-patch-cli/src/commands/vex_consumed.rs'
40+
- 'crates/socket-patch-cli/src/commands/vex_sources.rs'
41+
- 'crates/socket-patch-cli/src/commands/apply.rs'
42+
- 'crates/socket-patch-cli/tests/vex_e2e_common/**'
3743
- 'crates/socket-patch-cli/src/commands/scan/**'
3844
- 'crates/socket-patch-cli/src/commands/rollback.rs'
3945
- 'crates/socket-patch-cli/src/commands/vendor.rs'
@@ -60,6 +66,12 @@ on:
6066
- 'crates/socket-patch-core/src/vendor/**'
6167
- 'crates/socket-patch-core/src/patch/redirect/**'
6268
- 'crates/socket-patch-cli/src/commands/get.rs'
69+
- 'crates/socket-patch-core/src/vex/**'
70+
- 'crates/socket-patch-cli/src/commands/vex.rs'
71+
- 'crates/socket-patch-cli/src/commands/vex_consumed.rs'
72+
- 'crates/socket-patch-cli/src/commands/vex_sources.rs'
73+
- 'crates/socket-patch-cli/src/commands/apply.rs'
74+
- 'crates/socket-patch-cli/tests/vex_e2e_common/**'
6375
- 'crates/socket-patch-cli/src/commands/scan/**'
6476
- 'crates/socket-patch-cli/src/commands/rollback.rs'
6577
- 'crates/socket-patch-cli/src/commands/vendor.rs'

‎.github/workflows/ci.yml‎

Lines changed: 474 additions & 28 deletions
Large diffs are not rendered by default.
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
name: Go patch compatibility
2+
3+
# Real Go releases (1.18 = first with go.work, 1.21, 1.24 = the CI pin,
4+
# 1.26 = newest) through the hermetic hosted, vendored, go.work and
5+
# go-patches capstones, each ending in the manifest-less VEX step. The
6+
# suites use a file GOPROXY, so no network beyond the toolchain download.
7+
8+
on:
9+
pull_request:
10+
paths:
11+
- '.github/workflows/go-compatibility.yml'
12+
- 'crates/socket-patch-core/src/vendor/go*.rs'
13+
- 'crates/socket-patch-core/src/vendor/golang.rs'
14+
- 'crates/socket-patch-core/src/patch/redirect/**'
15+
- 'crates/socket-patch-core/src/crawlers/go_crawler.rs'
16+
- 'crates/socket-patch-core/src/vex/**'
17+
- 'crates/socket-patch-cli/src/commands/vex*.rs'
18+
- 'crates/socket-patch-cli/src/commands/vendor*'
19+
- 'crates/socket-patch-cli/tests/e2e_golang_*build.rs'
20+
- 'crates/socket-patch-cli/tests/e2e_vendor_golang_build.rs'
21+
- 'crates/socket-patch-cli/tests/golang_e2e_matrix/**'
22+
- 'crates/socket-patch-cli/tests/e2e_vex_lockfile/golang.rs'
23+
- 'crates/socket-patch-cli/tests/vex_e2e_common/**'
24+
push:
25+
branches: [main]
26+
workflow_dispatch:
27+
28+
permissions:
29+
contents: read
30+
31+
concurrency:
32+
group: go-compat-${{ github.event.pull_request.number || github.ref }}
33+
cancel-in-progress: true
34+
35+
env:
36+
SOCKET_NO_CONFIG: '1'
37+
SOCKET_NO_UPDATE_CHECK: '1'
38+
39+
jobs:
40+
go:
41+
strategy:
42+
fail-fast: false
43+
matrix:
44+
os: [ubuntu-latest]
45+
go: ['1.18.10', '1.21.13', '1.24.13', '1.26.3']
46+
include:
47+
# macOS-latest dyld refuses binaries without LC_UUID (Go < 1.24
48+
# linkers; see ci.yml's vexctl step).
49+
- {os: macos-latest, go: '1.24.13'}
50+
- {os: macos-latest, go: '1.26.3'}
51+
runs-on: ${{ matrix.os }}
52+
timeout-minutes: 45
53+
steps:
54+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
55+
with:
56+
persist-credentials: false
57+
- run: rustup show
58+
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
59+
with:
60+
key: go-compat
61+
save-if: ${{ github.ref == 'refs/heads/main' }}
62+
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
63+
with:
64+
go-version: ${{ matrix.go }}
65+
cache: false
66+
- name: Real-go hosted + vendored flows with manifest-less VEX
67+
env:
68+
SOCKET_PATCH_GO_E2E_VERSION: ${{ matrix.go }}
69+
SOCKET_PATCH_GO_E2E_REQUIRED: '1'
70+
# The hermetic manifest-less VEX cells are the `golang::` module of
71+
# the shared `e2e_vex_lockfile` binary; its filter goes in a second
72+
# command so it does not also filter the build suites' tests.
73+
run: |
74+
cargo test -p socket-patch-cli --no-fail-fast \
75+
--test e2e_golang_hosted_build --test e2e_vendor_golang_build \
76+
--test e2e_golang_build --test e2e_golang_workspace_build -- --nocapture
77+
cargo test -p socket-patch-cli --test e2e_vex_lockfile -- golang:: --nocapture
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
name: npm hosted/vendored compatibility
2+
3+
# Real npm releases (at least one per major, 6 through 12) through the
4+
# hermetic hosted (`e2e_redirect_npm_build`) and vendored
5+
# (`e2e_vendor_npm_build`) capstones, each flow ending in the manifest-less
6+
# VEX checks. Builds the CLI and both test binaries once; every matrix leg
7+
# installs one pinned npm and runs them. See docs/testing/npm-compatibility.md.
8+
9+
on:
10+
pull_request:
11+
push:
12+
branches: [main]
13+
workflow_dispatch:
14+
15+
permissions:
16+
contents: read
17+
18+
jobs:
19+
build:
20+
runs-on: ubuntu-latest
21+
timeout-minutes: 25
22+
steps:
23+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
24+
with:
25+
persist-credentials: false
26+
- name: Compile CLI and npm capstones once
27+
run: |
28+
cargo test --locked -p socket-patch-cli --test e2e_redirect_npm_build --test e2e_vendor_npm_build --no-run --message-format=json > target-build.json
29+
python3 - <<'PY'
30+
import json, pathlib, shutil
31+
dest = pathlib.Path('target/npm-e2e'); dest.mkdir(parents=True, exist_ok=True)
32+
shutil.copy2('target/debug/socket-patch', dest / 'socket-patch')
33+
for line in pathlib.Path('target-build.json').read_text().splitlines():
34+
item = json.loads(line)
35+
name = item.get('target', {}).get('name')
36+
if name in ('e2e_redirect_npm_build', 'e2e_vendor_npm_build') and item.get('executable'):
37+
shutil.copy2(item['executable'], dest / name)
38+
assert all((dest / n).is_file() for n in ('e2e_redirect_npm_build', 'e2e_vendor_npm_build'))
39+
PY
40+
- uses: ./.github/actions/upload-artifact
41+
with:
42+
name: npm-e2e
43+
path: target/npm-e2e/
44+
if-no-files-found: error
45+
retention-days: 7
46+
47+
install-proof:
48+
needs: build
49+
runs-on: ubuntu-latest
50+
timeout-minutes: 25
51+
strategy:
52+
fail-fast: false
53+
matrix:
54+
include:
55+
# node 16's bundled npm 8 writes the v2 lock for the npm 6 x v2 test.
56+
- {npm: '6.14.18', node: '16.20.2'}
57+
- {npm: '7.0.0', node: '16.20.2'}
58+
- {npm: '7.24.2', node: '16.20.2'}
59+
- {npm: '8.19.4', node: '16.20.2'}
60+
- {npm: '9.0.0', node: '18.20.8'}
61+
- {npm: '9.9.4', node: '18.20.8'}
62+
- {npm: '10.9.9', node: '22.22.2'}
63+
- {npm: '11.20.0', node: '24.11.1'}
64+
- {npm: '12.0.0', node: '24.15.0'}
65+
- {npm: '12.1.0', node: '24.15.0'}
66+
steps:
67+
# Same workspace path as the build job, so the socket-patch path
68+
# compiled into the tests (CARGO_BIN_EXE_socket-patch) resolves.
69+
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
70+
with:
71+
pattern: npm-e2e*
72+
merge-multiple: true
73+
path: target/debug
74+
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
75+
with:
76+
node-version: '24.11.1'
77+
- name: Install the pinned npm
78+
env:
79+
NPM_TEST_VERSION: ${{ matrix.npm }}
80+
run: |
81+
npm install --prefix "$RUNNER_TEMP/npm-tool" --no-audit --no-fund "npm@$NPM_TEST_VERSION"
82+
echo "SOCKET_PATCH_NPM_E2E_BIN=$RUNNER_TEMP/npm-tool/node_modules/.bin/npm" >> "$GITHUB_ENV"
83+
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
84+
with:
85+
node-version: ${{ matrix.node }}
86+
- name: Hosted + vendored installs, manifest-less VEX
87+
env:
88+
SOCKET_PATCH_NPM_E2E_VERSION: ${{ matrix.npm }}
89+
SOCKET_PATCH_NPM_E2E_REQUIRED: '1'
90+
SOCKET_NO_CONFIG: '1'
91+
SOCKET_NO_UPDATE_CHECK: '1'
92+
run: |
93+
chmod +x target/debug/socket-patch target/debug/e2e_redirect_npm_build target/debug/e2e_vendor_npm_build
94+
target/debug/e2e_redirect_npm_build --include-ignored --test-threads 4
95+
target/debug/e2e_vendor_npm_build --include-ignored --test-threads 4

‎.github/workflows/pdm-compatibility.yml‎

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,11 @@ on:
2020
- 'crates/socket-patch-cli/src/commands/scan/**'
2121
- 'crates/socket-patch-cli/src/commands/rollback.rs'
2222
- 'crates/socket-patch-core/tests/fixtures/pdm-native/**'
23+
- 'crates/socket-patch-core/src/vex/**'
24+
- 'crates/socket-patch-cli/src/commands/vex*.rs'
25+
- 'crates/socket-patch-cli/tests/e2e_vex_build/main.rs'
26+
- 'crates/socket-patch-cli/tests/e2e_vex_build/pdm.rs'
27+
- 'crates/socket-patch-cli/tests/vex_pypi_real_common/**'
2328
push:
2429
branches: [main]
2530
paths:
@@ -83,7 +88,7 @@ jobs:
8388
# Every stable PDM major family (0.x, 1.x, 2.x) and each 2.x lock-format
8489
# boundary, on Linux and Windows; macOS samples the ends of the range.
8590
os: [ubuntu-latest, windows-latest]
86-
pdm: ['0.12.3', '1.15.5', '2.0.3', '2.1.5', '2.3.4', '2.6.1', '2.7.4', '2.8.2', '2.9.3', '2.10.4', '2.11.2', '2.17.3', '2.20.1', '2.25.9', '2.29.2']
91+
pdm: ['0.12.3', '1.15.5', '2.0.3', '2.1.5', '2.3.4', '2.6.1', '2.7.4', '2.8.2', '2.9.3', '2.10.4', '2.11.2', '2.17.3', '2.20.1', '2.22.4', '2.25.9', '2.29.2']
8792
include:
8893
- { os: macos-latest, pdm: '0.12.3' }
8994
- { os: macos-latest, pdm: '2.8.2' }
@@ -143,3 +148,35 @@ jobs:
143148
native-pdm/cases/**/pdm.lock
144149
native-pdm/cases/**/*.log
145150
retention-days: 14
151+
152+
# The hermetic Rust capstone (wiremock Socket API that also serves the
153+
# hosted wheel) over every PDM release the backtest covers: real hosted +
154+
# vendored flows ending in the manifest-less VEX matrix; refused lock
155+
# formats (3.1, 4.0-4.2) must attest nothing.
156+
capstone:
157+
strategy:
158+
fail-fast: false
159+
matrix:
160+
os: [ubuntu-latest, macos-latest]
161+
pdm: ['0.12.3', '1.0.0', '1.4.5', '1.8.5', '1.15.5', '2.0.3', '2.7.4', '2.8.2', '2.10.4', '2.11.2', '2.17.3', '2.20.1', '2.22.4', '2.25.9', '2.29.2']
162+
runs-on: ${{ matrix.os }}
163+
timeout-minutes: 30
164+
steps:
165+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
166+
with:
167+
persist-credentials: false
168+
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
169+
with:
170+
key: pdm-vex-capstone
171+
# Only main writes the cache: 30 PR matrix cells saving would churn
172+
# the repo's 10 GiB budget (ci.yml's rust-cache note).
173+
save-if: ${{ github.ref == 'refs/heads/main' }}
174+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
175+
with:
176+
python-version: '3.12'
177+
- run: python -m pip install uv==0.11.19
178+
- name: Real-PDM hosted + vendored flows with manifest-less VEX
179+
env:
180+
SOCKET_PATCH_PDM_E2E_REQUIRED: '1'
181+
SOCKET_PATCH_PDM_E2E_VERSION: ${{ matrix.pdm }}
182+
run: cargo test --locked -p socket-patch-cli --test e2e_vex_build -- 'pdm::' --ignored

‎.github/workflows/pipenv-compatibility.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@ on:
2323
- 'crates/socket-patch-cli/src/commands/vex.rs'
2424
- 'crates/socket-patch-core/src/patch/redirect/replay.rs'
2525
- 'scripts/backtest-pipenv.py'
26+
- 'scripts/tests/test_backtest_harnesses.py'
27+
- 'crates/socket-patch-core/src/vex/**'
28+
- 'crates/socket-patch-cli/src/commands/vex_sources.rs'
2629
- '.github/workflows/pipenv-compatibility.yml'
2730
push:
2831
branches: [main]
@@ -70,6 +73,8 @@ jobs:
7073
cargo build --locked -p socket-patch-cli
7174
mkdir -p "$RUNNER_TEMP/bin"
7275
cp target/debug/socket-patch "$RUNNER_TEMP/bin/socket-patch"
76+
- name: Harness unit tests
77+
run: python3 -m unittest scripts/tests/test_backtest_harnesses.py
7378
- name: Run the Pipenv matrix
7479
env:
7580
SOCKET_NO_CONFIG: '1'

‎.github/workflows/pnpm-compatibility.yml‎

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,19 +17,22 @@ jobs:
1717
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
1818
with:
1919
persist-credentials: false
20-
- name: Compile CLI and pnpm integration test once
20+
- name: Compile CLI and pnpm integration tests once
2121
run: |
22-
cargo test --locked -p socket-patch-cli --test e2e_redirect_pnpm_build --no-run --message-format=json > target-build.json
22+
cargo test --locked -p socket-patch-cli --test e2e_redirect_pnpm_build --test e2e_vendor_pnpm_build --no-run --message-format=json > target-build.json
2323
python3 - <<'PY'
2424
import json, pathlib, shutil
2525
dest = pathlib.Path('target/pnpm-e2e')
2626
dest.mkdir(parents=True, exist_ok=True)
2727
shutil.copy2('target/debug/socket-patch', dest / 'socket-patch')
28+
names = {'e2e_redirect_pnpm_build': 'pnpm-e2e', 'e2e_vendor_pnpm_build': 'pnpm-vendor-e2e'}
2829
for line in pathlib.Path('target-build.json').read_text().splitlines():
2930
item = json.loads(line)
30-
if item.get('target', {}).get('name') == 'e2e_redirect_pnpm_build' and item.get('executable'):
31-
shutil.copy2(item['executable'], dest / 'pnpm-e2e')
32-
assert (dest / 'pnpm-e2e').is_file()
31+
name = item.get('target', {}).get('name')
32+
if name in names and item.get('executable'):
33+
shutil.copy2(item['executable'], dest / names[name])
34+
for n in names.values():
35+
assert (dest / n).is_file(), n
3336
PY
3437
- uses: ./.github/actions/upload-artifact
3538
with:
@@ -41,7 +44,7 @@ jobs:
4144
install-proof:
4245
needs: build
4346
runs-on: ubuntu-latest
44-
timeout-minutes: 10
47+
timeout-minutes: 15
4548
strategy:
4649
fail-fast: false
4750
matrix:
@@ -100,3 +103,15 @@ jobs:
100103
chmod +x bin/socket-patch bin/pnpm-e2e
101104
export SOCKET_PATCH_PNPM_E2E_SOCKET_BIN="$PWD/bin/socket-patch"
102105
bin/pnpm-e2e pnpm_pinned_matrix --ignored --nocapture
106+
- name: Require vendored lifecycle and manifest-less VEX
107+
# pnpm >= 9: full vendored capstone (vendor + get --mode vendored);
108+
# 7-8: the legacy lifecycle; 1-6: vendoring refused, nothing attested.
109+
env:
110+
SOCKET_PATCH_PNPM_E2E_VERSION: ${{ matrix.pnpm }}
111+
SOCKET_PATCH_PNPM_E2E_REQUIRED: '1'
112+
SOCKET_NO_CONFIG: '1'
113+
SOCKET_NO_UPDATE_CHECK: '1'
114+
run: |
115+
chmod +x bin/socket-patch bin/pnpm-vendor-e2e
116+
export SOCKET_PATCH_PNPM_E2E_SOCKET_BIN="$PWD/bin/socket-patch"
117+
bin/pnpm-vendor-e2e pnpm_pinned_matrix --ignored --nocapture

0 commit comments

Comments
 (0)