Skip to content

Commit b41b9d3

Browse files
Fix Pipenv ignoring pylock patch wiring (#912, #1122) (#1193)
* Start fix for #912, #1122 Assisted-by: Claude Code:claude-opus-5-5 * Stop wiring pylocks that Pipenv installs from A pylock.toml (or pylock.<name>.toml) beside a Pipfile is read by Pipenv, not by a PEP 751 installer, and every pylock code path treated it as installer-neutral: - Vendored scans of a `use_pylock = true` project (Pipfile, Pipfile.lock and pylock.toml) wired only pylock.toml. Pipenv installs from Pipfile.lock when both exist, so every `pipenv sync` and `install --deploy` got the unpatched release after a "success" run (#1122). The router now wires Pipfile.lock and names the pylock in the pypi_multiple_lockfiles warning. - Hosted and vendored scans of a pylock-only Pipenv checkout wrote an `archive` entry that Pipenv's pylock reader drops, so `pipenv sync` installed the upstream release while the scan reported success, and vendored VEX attested not_affected (#912). Both modes now refuse with a `pipenv lock` pointer (redirect_pipenv_pylock_unsupported / pypi_pipenv_pylock_unsupported), and VEX discovery no longer trusts a Socket reference in such a pylock. The rule lives in one predicate, utils::python_lock::pipenv_reads_pylock, shared by the hosted rewriter, the vendored router and VEX discovery. Assisted-by: Claude Code:claude-opus-5-5 * Test Pipenv pylock projects with real Pipenv Adds a real-Pipenv 2026 e2e (part of the existing e2e_vex_build pipenv:: legs) for a `[pipenv] use_pylock = true` project: - with Pipfile.lock and pylock.toml, vendored mode wires Pipfile.lock and a fresh `pipenv install --deploy` gets the patched release (#1122); - the pylock-only checkout is refused in hosted and vendored mode and the pylock is left untouched (#912). Documents both cases in the Pipenv compatibility table. Assisted-by: Claude Code:claude-opus-5-5 * Let a governing tool lock outrank a stray Pipfile A Pipfile left beside a governing uv.lock, poetry.lock or pdm.lock does not make Pipenv the installer, but the hosted pylock rewriter and VEX discovery still treated the pylock as Pipenv's. The hosted refusal then marked the patch refused for every Python lock, so a successful uv.lock pin was reported as not redirected and dropped from VEX. pipenv_reads_pylock now applies the same tool-lock precedence the vendored router already used, so all three call sites agree. Assisted-by: Claude Code:claude-opus-5-5 * Serialize real-PM tool venv bootstraps bootstrap_tool checks for a healthy tool venv and then creates it, with no lock in between. The Pipenv e2e suite now has two tests, and libtest runs them in parallel. On a cold tools cache both miss the probe and race `uv venv` on the same pipenv-<version> directory, so one fails with "a virtual environment already exists". This reproduced in CI and 5/5 times locally from an empty tools root. A process-wide lock around the check-and-create fixes it; warm-cache runs only pay for a --version probe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PzAvnBX6HySvR6zXsFtdy7 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 5ab2b64 commit b41b9d3

8 files changed

Lines changed: 636 additions & 9 deletions

File tree

‎crates/socket-patch-cli/tests/e2e_vex_build/pipenv.rs‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -350,3 +350,136 @@ fn pipenv_every_major_hosted_and_vendored_end_in_manifest_less_vex() {
350350
}
351351
assert!(failures.is_empty(), "{}", failures.join("\n\n"));
352352
}
353+
354+
const PYLOCK_PIPFILE: &str = "[[source]]\nurl = \"https://pypi.org/simple\"\nverify_ssl = true\nname = \"pypi\"\n\n[packages]\nsix = \"==1.16.0\"\n\n[dev-packages]\n\n[pipenv]\nuse_pylock = true\n";
355+
356+
/// #912 / #1122 on a real Pipenv 2026 `use_pylock = true` project, whose
357+
/// `pipenv lock` writes both `Pipfile.lock` and `pylock.toml`:
358+
///
359+
/// * #1122: vendored wires `Pipfile.lock` (what Pipenv installs from when
360+
/// both exist), leaves the pylock alone, and a fresh
361+
/// `pipenv install --deploy` gets the PATCHED six.
362+
/// * #912: the pylock-only checkout (no `Pipfile.lock`) is refused in both
363+
/// modes — Pipenv would drop the `archive` entry and install the upstream
364+
/// release — and the pylock is left byte-identical.
365+
fn pylock_flow(pipenv: &Pipenv, root: &Path) {
366+
let v = pipenv.version.as_str();
367+
let what = format!("pipenv {v} use_pylock");
368+
let proj = root.join("pylock").join("proj");
369+
std::fs::create_dir_all(&proj).unwrap();
370+
std::fs::write(proj.join("Pipfile"), PYLOCK_PIPFILE).unwrap();
371+
let python = venv_bin(&pipenv.venv, "python");
372+
pipenv.project_venv(&proj);
373+
let out = pipenv.run(&proj, &["install", "--python", python.to_str().unwrap()]);
374+
assert_ok(&out, &format!("{what}: pipenv install"));
375+
if !proj.join("pylock.toml").is_file() {
376+
assert_ok(
377+
&pipenv.run(&proj, &["lock"]),
378+
&format!("{what}: pipenv lock"),
379+
);
380+
}
381+
let pristine_pylock = read(&proj.join("pylock.toml"));
382+
let py = venv_bin(&proj.join(".venv"), "python");
383+
let (_, pristine, _) =
384+
six_oracle(&py, &proj).unwrap_or_else(|| panic!("{what}: six not importable"));
385+
let patched = [pristine.as_slice(), PATCH_SUFFIX].concat();
386+
387+
// #912: the pylock-only checkout, both modes.
388+
for mode in [Mode::Hosted, Mode::Vendored] {
389+
let only = root.join("pylock").join(format!("only-{}", mode.label()));
390+
copy_tree(&proj, &only, &[".venv"]);
391+
std::fs::remove_file(only.join("Pipfile.lock")).unwrap();
392+
let api = RealApi::start(mode.uuid(), &pristine, &patched);
393+
let (_, env, stderr) = socket_scan(&only, &api, mode.scan_flags(), &pipenv.envs);
394+
let code = match mode {
395+
Mode::Hosted => "redirect_pipenv_pylock_unsupported",
396+
Mode::Vendored => "pypi_pipenv_pylock_unsupported",
397+
};
398+
let ok = env.to_string().contains(code)
399+
&& read(&only.join("pylock.toml")) == pristine_pylock
400+
&& !only.join("Pipfile.lock").exists();
401+
record(
402+
"pipenv",
403+
v,
404+
&format!("pylock-only/{}", mode.label()),
405+
"refused",
406+
if ok { "pass" } else { "FAIL" },
407+
);
408+
assert!(
409+
ok,
410+
"{what} pylock-only {}: expected {code}, pylock untouched: {env}\n{stderr}",
411+
mode.label()
412+
);
413+
}
414+
415+
// #1122: both locks, vendored.
416+
let mode = Mode::Vendored;
417+
let api = RealApi::start(mode.uuid(), &pristine, &patched);
418+
let (code, env, stderr) = socket_scan(&proj, &api, mode.scan_flags(), &pipenv.envs);
419+
assert_eq!(code, Some(0), "{what}: vendored scan: {env}\n{stderr}");
420+
let lock = String::from_utf8(read(&proj.join("Pipfile.lock"))).unwrap();
421+
assert!(
422+
lock.contains(&format!(".socket/vendor/pypi/{}/", mode.uuid())),
423+
"{what}: Pipfile.lock must point at the vendored wheel: {lock}"
424+
);
425+
assert_eq!(
426+
read(&proj.join("pylock.toml")),
427+
pristine_pylock,
428+
"{what}: the pylock Pipenv ignores is left alone"
429+
);
430+
let fresh = root.join("pylock").join("fresh");
431+
copy_tree(&proj, &fresh, &[".venv"]);
432+
pipenv.project_venv(&fresh);
433+
let out = pipenv.run(
434+
&fresh,
435+
&["install", "--deploy", "--python", python.to_str().unwrap()],
436+
);
437+
assert_ok(&out, &format!("{what}: fresh `pipenv install --deploy`"));
438+
let (_, bytes, is_patched) = six_oracle(&venv_bin(&fresh.join(".venv"), "python"), &fresh)
439+
.unwrap_or_else(|| panic!("{what}: six not importable in the fresh checkout"));
440+
let ok = is_patched && bytes == patched;
441+
record(
442+
"pipenv",
443+
v,
444+
"use_pylock/vendored",
445+
"install-patched",
446+
if ok { "pass" } else { "FAIL" },
447+
);
448+
assert!(ok, "{what}: the fresh install must carry the PATCHED six");
449+
}
450+
451+
#[test]
452+
#[ignore = "real Pipenv releases + PyPI (network). Run with --ignored."]
453+
fn pipenv_pylock_projects_wire_what_pipenv_installs() {
454+
let Some(uv) = find_uv() else {
455+
return skip_or_fail(REQUIRED, "uv is not installed");
456+
};
457+
let mut failures = Vec::new();
458+
// `[pipenv] use_pylock` is a Pipenv 2026 setting.
459+
for version in versions(VERSIONS_VAR, VERSIONS)
460+
.into_iter()
461+
.filter(|v| year(v) >= 2026)
462+
{
463+
let scratch = tempfile::tempdir().unwrap();
464+
let pipenv = match Pipenv::bootstrap(&uv, &version, scratch.path()) {
465+
Ok(p) => p,
466+
Err(e) => {
467+
record("pipenv", &version, "use_pylock", "bootstrap", "SKIP");
468+
skip_or_fail(REQUIRED, &format!("pipenv {version} bootstrap: {e}"));
469+
continue;
470+
}
471+
};
472+
let root = scratch.path().join("run");
473+
let result =
474+
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| pylock_flow(&pipenv, &root)));
475+
if let Err(e) = result {
476+
let msg = e
477+
.downcast_ref::<String>()
478+
.cloned()
479+
.or_else(|| e.downcast_ref::<&str>().map(|s| s.to_string()))
480+
.unwrap_or_default();
481+
failures.push(format!("pipenv {version} use_pylock: {msg}"));
482+
}
483+
}
484+
assert!(failures.is_empty(), "{}", failures.join("\n\n"));
485+
}

‎crates/socket-patch-cli/tests/vex_pipenv_pip_real/mod.rs‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -268,14 +268,18 @@ pub fn make_venv(uv: &Path, python: &str, venv: &Path, pins: &[&str]) -> Result<
268268
}
269269

270270
/// `<tools_root>/<name>` made by [`make_venv`] unless `probe` already runs
271-
/// there (bootstraps are reused across runs).
271+
/// there (bootstraps are reused across runs). Serialized: tests in one
272+
/// binary run in parallel, and two of them bootstrapping the same venv
273+
/// race `uv venv` into "a virtual environment already exists".
272274
pub fn bootstrap_tool(
273275
uv: &Path,
274276
name: &str,
275277
python: &str,
276278
pins: &[&str],
277279
probe: &str,
278280
) -> Result<PathBuf, String> {
281+
static BOOTSTRAP: std::sync::Mutex<()> = std::sync::Mutex::new(());
282+
let _guard = BOOTSTRAP.lock().unwrap_or_else(|e| e.into_inner());
279283
let venv = tools_root().join(name);
280284
let exe = venv_bin(&venv, probe);
281285
let healthy = |exe: &Path| {

‎crates/socket-patch-core/src/patch/redirect/mod.rs‎

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ mod lock_index_equivalence_tests;
5050
pub mod npmrc;
5151
mod pdm;
5252
mod pipenv;
53+
#[cfg(test)]
54+
mod pipenv_pylock_tests;
5355
pub mod presence;
5456
// The pnpm hosted planner lives with the format's model.
5557
use crate::formats::cargo::hosted::CargoLockPlan;
@@ -5331,6 +5333,24 @@ fn record_python_metadata_edit(
53315333
result.files.insert(edit.path, edit.rewritten);
53325334
}
53335335

5336+
/// #912: the refusal for a pylock Pipenv installs from
5337+
/// ([`pipenv_reads_pylock`]): Pipenv drops the `archive` entry a pin
5338+
/// writes, so `pipenv sync` would install the upstream release.
5339+
///
5340+
/// [`pipenv_reads_pylock`]: crate::utils::python_lock::pipenv_reads_pylock
5341+
fn pipenv_pylock_refusal(path: &str, dep: &DepOverride) -> RewriteWarning {
5342+
RewriteWarning {
5343+
code: "redirect_pipenv_pylock_unsupported".into(),
5344+
detail: format!(
5345+
"{path} is installed by Pipenv (a Pipfile sits beside it and there is no \
5346+
Pipfile.lock), and Pipenv keeps only the version and hashes of a pylock \
5347+
entry, so a pin of {}=={} would be dropped and the upstream release \
5348+
installed; run `pipenv lock` to write a Pipfile.lock and re-run the scan",
5349+
dep.name, dep.version
5350+
),
5351+
}
5352+
}
5353+
53345354
/// Each lock is parsed once ([`PythonLockSession`]) and every dep is
53355355
/// planned, refused or applied against that one document. The lock is still
53365356
/// rendered after every rewritten dep: each dep's FileEdit fragments are
@@ -5343,7 +5363,9 @@ fn rewrite_uv_lock(
53435363
python_metadata: &BTreeMap<String, String>,
53445364
result: &mut RewriteResult,
53455365
) {
5346-
use crate::utils::python_lock::{is_python_lock_name, ArtifactSource, PythonLockSession};
5366+
use crate::utils::python_lock::{
5367+
is_python_lock_name, pipenv_reads_pylock, ArtifactSource, PythonLockSession,
5368+
};
53475369

53485370
let locks: Vec<(&String, &String)> = files
53495371
.iter()
@@ -5369,9 +5391,17 @@ fn rewrite_uv_lock(
53695391
for (path, original) in locks {
53705392
let mut content = original.clone();
53715393
let mut session = PythonLockSession::new(original);
5394+
let pipenv_reads = pipenv_reads_pylock(path, |rel| files.contains_key(rel));
53725395
for &(dep, sha256) in &usable {
53735396
let artifact = ArtifactSource::Url(&dep.artifact_url);
53745397
let plan = match session.plan(&content, &dep.name, &dep.version, artifact) {
5398+
Ok(Some(_)) if pipenv_reads => {
5399+
result
5400+
.refused_python_lock_uuids
5401+
.insert(dep.patch_uuid.clone());
5402+
result.warnings.push(pipenv_pylock_refusal(path, dep));
5403+
continue;
5404+
}
53755405
Ok(Some(plan)) => plan,
53765406
Ok(None) => {
53775407
result.warnings.push(RewriteWarning {
Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
//! #912 / #1122: a PEP 751 `pylock*.toml` beside a `Pipfile` is read by
2+
//! Pipenv, not by a PEP 751 installer. Pipenv's pylock reader
3+
//! (`PylockFile.convert_to_pipenv_lockfile`) keeps only each package's
4+
//! version, marker and wheel / sdist hashes, so the `archive` entry a hosted
5+
//! pin writes is dropped and `pipenv sync` installs the upstream release.
6+
//! With no `Pipfile.lock` (which Pipenv prefers when both exist) the pin is
7+
//! refused; with one, both locks are pinned as before.
8+
9+
use super::*;
10+
11+
const WHEEL: &str = "six-1.16.0-py2.py3-none-any.whl";
12+
const UUID: &str = "aaaaaaaa-0000-4000-8000-000000000912";
13+
const HEX: &str = "8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254";
14+
const PIPFILE: &str = "[[source]]\nurl = \"https://pypi.org/simple\"\nverify_ssl = true\nname = \"pypi\"\n\n[packages]\nsix = \"==1.16.0\"\n\n[pipenv]\nuse_pylock = true\n";
15+
16+
fn dep() -> DepOverride {
17+
DepOverride {
18+
ecosystem: "pypi".into(),
19+
name: "six".into(),
20+
namespace: None,
21+
version: "1.16.0".into(),
22+
token: "11111111-1111-4111-8111-111111111111".into(),
23+
patch_uuid: UUID.into(),
24+
artifact_url: format!(
25+
"https://patch.socket.dev/patch/pypi/six/1.16.0/11111111-1111-4111-8111-111111111111/{UUID}/{WHEEL}"
26+
),
27+
registry_override: None,
28+
integrity: Integrity {
29+
sha256: Some(HEX.into()),
30+
..Default::default()
31+
},
32+
}
33+
}
34+
35+
/// A pylock as `pipenv lock` (use_pylock = true) writes it.
36+
fn pylock() -> String {
37+
format!(
38+
"lock-version = \"1.0\"\nenvironments = []\nextras = []\ndependency-groups = []\ndefault-groups = []\ncreated-by = \"pipenv\"\n\n[[packages]]\nname = \"six\"\nversion = \"1.16.0\"\nindex = \"https://pypi.org/simple\"\nwheels = [{{ name = \"{WHEEL}\", url = \"https://files.pythonhosted.org/packages/d9/5a/{WHEEL}\", hashes = {{ sha256 = \"{HEX}\" }} }}]\n\n[tool.pipenv]\ngenerated_from = \"Pipfile.lock\"\n"
39+
)
40+
}
41+
42+
fn pipfile_lock() -> String {
43+
format!(
44+
"{{\n \"_meta\": {{\n \"hash\": {{\n \"sha256\": \"unchanged\"\n }},\n \"pipfile-spec\": 6,\n \"sources\": [\n {{\n \"name\": \"pypi\",\n \"url\": \"https://pypi.org/simple\",\n \"verify_ssl\": true\n }}\n ]\n }},\n \"default\": {{\n \"six\": {{\n \"hashes\": [\n \"sha256:{HEX}\"\n ],\n \"index\": \"pypi\",\n \"version\": \"==1.16.0\"\n }}\n }},\n \"develop\": {{}}\n}}\n"
45+
)
46+
}
47+
48+
fn files(entries: &[(&str, String)]) -> BTreeMap<String, String> {
49+
entries
50+
.iter()
51+
.map(|(k, v)| (k.to_string(), v.clone()))
52+
.collect()
53+
}
54+
55+
fn rewrite(files: &BTreeMap<String, String>) -> RewriteResult {
56+
rewrite_registry_redirect_with_pipenv_version(
57+
files,
58+
&[dep()],
59+
&BTreeMap::new(),
60+
Some(2026),
61+
false,
62+
)
63+
}
64+
65+
/// #912: a pylock-only Pipenv checkout (any `pylock_name`) is refused,
66+
/// loudly, and nothing is written or confirmed for it.
67+
#[test]
68+
fn pylock_read_by_pipenv_is_refused_without_pipfile_lock() {
69+
for (pipfile, lock) in [("Pipfile", "pylock.toml"), ("Pipfile", "pylock.dev.toml")] {
70+
let result = rewrite(&files(&[(pipfile, PIPFILE.into()), (lock, pylock())]));
71+
assert!(
72+
result.files.is_empty() && result.edits.is_empty(),
73+
"{lock}: pinned {:?}",
74+
result.files.keys().collect::<Vec<_>>()
75+
);
76+
assert!(
77+
!result.confirmed_python_lock_uuids.contains(UUID),
78+
"{lock}: confirmed"
79+
);
80+
assert!(result.refused_python_lock_uuids.contains(UUID), "{lock}");
81+
let refusals: Vec<&RewriteWarning> = result
82+
.warnings
83+
.iter()
84+
.filter(|w| w.code == "redirect_pipenv_pylock_unsupported")
85+
.collect();
86+
assert_eq!(refusals.len(), 1, "{lock}: {:?}", result.warnings);
87+
assert!(
88+
refusals[0].detail.contains(lock)
89+
&& refusals[0].detail.contains("pipenv lock")
90+
&& refusals[0].detail.contains("six==1.16.0"),
91+
"{lock}: {}",
92+
refusals[0].detail
93+
);
94+
}
95+
}
96+
97+
/// Controls: a pylock with no Pipfile beside it is pinned as before, and a
98+
/// `use_pylock = true` project with both locks still pins both (#1122's
99+
/// hosted control).
100+
#[test]
101+
fn pylock_without_a_pipenv_consumer_or_beside_pipfile_lock_still_pins() {
102+
let result = rewrite(&files(&[("pylock.toml", pylock())]));
103+
assert!(
104+
result
105+
.files
106+
.get("pylock.toml")
107+
.is_some_and(|t| t.contains(WHEEL) && t.contains("archive")),
108+
"{:?}",
109+
result.warnings
110+
);
111+
assert!(result.confirmed_python_lock_uuids.contains(UUID));
112+
113+
let result = rewrite(&files(&[
114+
("Pipfile", PIPFILE.into()),
115+
("Pipfile.lock", pipfile_lock()),
116+
("pylock.toml", pylock()),
117+
]));
118+
for lock in ["Pipfile.lock", "pylock.toml"] {
119+
assert!(
120+
result.files.get(lock).is_some_and(|t| t.contains(UUID)),
121+
"{lock}: {:?}",
122+
result.warnings
123+
);
124+
}
125+
assert!(result
126+
.warnings
127+
.iter()
128+
.all(|w| w.code != "redirect_pipenv_pylock_unsupported"));
129+
}
130+
131+
/// A leftover `Pipfile` beside a governing uv / Poetry / PDM lock does not
132+
/// make Pipenv the installer: the pylock is pinned as before, and no Pipenv
133+
/// refusal vetoes the sibling lock's confirmation.
134+
#[test]
135+
fn pylock_beside_a_governing_tool_lock_is_not_pipenvs() {
136+
let uv_lock = format!(
137+
"version = 1\nrevision = 3\nrequires-python = \">=3.9\"\n\n[[package]]\nname = \"six\"\nversion = \"1.16.0\"\nsource = {{ registry = \"https://pypi.org/simple\" }}\nwheels = [{{ url = \"https://files.pythonhosted.org/packages/d9/5a/{WHEEL}\", hash = \"sha256:{HEX}\" }}]\n"
138+
);
139+
for (lock, text) in [
140+
("uv.lock", uv_lock),
141+
("poetry.lock", String::new()),
142+
("pdm.lock", String::new()),
143+
] {
144+
let result = rewrite(&files(&[
145+
("Pipfile", PIPFILE.into()),
146+
(lock, text),
147+
("pylock.toml", pylock()),
148+
]));
149+
assert!(
150+
result
151+
.warnings
152+
.iter()
153+
.all(|w| w.code != "redirect_pipenv_pylock_unsupported"),
154+
"{lock}: {:?}",
155+
result.warnings
156+
);
157+
// Only the real uv.lock fixture pins cleanly; the empty Poetry /
158+
// PDM stubs are refused by their own rewriters.
159+
if lock != "uv.lock" {
160+
continue;
161+
}
162+
assert!(
163+
!result.refused_python_lock_uuids.contains(UUID),
164+
"{lock}: {:?}",
165+
result.warnings
166+
);
167+
assert!(
168+
result
169+
.files
170+
.get("pylock.toml")
171+
.is_some_and(|t| t.contains(WHEEL) && t.contains("archive")),
172+
"{lock}: {:?}",
173+
result.warnings
174+
);
175+
}
176+
}

0 commit comments

Comments
 (0)