Skip to content

fix(reproduce): preserve user files when cleaning download caches - #367

Open
reacher-z wants to merge 1 commit into
mainfrom
codex/reproduce-owned-cache
Open

reacher-z wants to merge 1 commit into
mainfrom
codex/reproduce-owned-cache

Conversation

@reacher-z

Copy link
Copy Markdown
Collaborator

What does this PR do?

clawbench-reproduce --work-dir <existing-directory> previously deleted the entire supplied directory after comparison, including unrelated user files. Downloads now go into a unique invocation-owned child directory. Only that child is removed when the invocation finishes or unwinds; --keep-cache retains it and prints its exact path.

Temporary-directory cleanup covers PASS, FAIL, download errors, judge errors and interruption. Independent child directories prevent one invocation from deleting another invocation's cache. The caller's parent directory and pre-existing contents are preserved. Includes the required Unreleased changelog entry.

Corpus

  • v2
  • v1
  • both
  • not applicable

Host-side reproduction CLI cleanup only.

Test plan

  • 11 new offline regressions: five completion/failure modes with and without --keep-cache, plus overlapping cache lifetimes. Each CLI case preserves an unrelated sentinel and a previous cache.
  • uv run --frozen pytest -q — 294 passed, 3 skipped.
  • Ruff check/format on changed Python files; uv run --frozen pyright — 0 errors.
  • git diff --check.
  • uv build — wheel and sdist successfully built.

Download and judge calls are mocked in the regression tests; no live HF/model run or package publication was performed. Reproduction metric validation and judge-cache correctness remain separate issues #362#364. Retained downloads now live one unique directory below --work-dir, so callers inspecting the old direct layout should use the printed cache path.

Related issues

Fixes #361. Part of the next-release quality checklist #366.

@Perry2004 please review the cache ownership/cleanup behavior and prioritize this data-loss fix for the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] reproduce: cleanup deletes unrelated files in the user-supplied work directory

1 participant