Halide server by RMPR · Pull Request #7 · zivid/halide-cache

RMPR · 2026-09-20T15:25:15Z

No description provided.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

Prefix the hash input with a key scheme version, mix in the host OS and architecture, hash an optional --builder-id, and replace the whole-argument base_dir stripping with a CTCACHE_STRIP-like substring stripper that also accepts extra --strip prefixes and normalises path separators. Also fix tests/run, which still relied on the removed HALIDE_CACHE_DIR environment variable. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

Add a Kind enum for file vs directory entries, open_raw/store_raw to copy the compressed blob without recompressing (temp file plus rename), and make the LRU scan skip in-progress temporary files. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

An HTTP server sharing cache entries between machines. Blobs are stored via lager with the same layout as the local cache. GET/HEAD/PUT /v1/blobs/{addr}, GET /v1/stats and /healthz. Uploads optionally require a bearer token. The on-disk size is bounded by --size (default 40 GiB) through a periodic LRU pass that also runs after 5 % worth of new uploads. Ships a systemd unit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

Entries missing locally are fetched from the server into the local cache and new entries are uploaded after a build, unless --remote-read-only is given. Uploads carry the --remote-token as a bearer token. Any failure talking to the server is a warning and the build proceeds locally. A partial remote hit is treated as a miss. Extend tests/run with an end-to-end scenario starting the server. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

Track per-minute history and per-client-IP statistics in memory, expose them at /v1/history and /v1/clients, and serve a self-contained HTML dashboard at / with stat tiles, charts over time and a client table. CI: run the end-to-end test as its own step. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

The client sends its hostname in an X-Halide-Cache-Client header; the server keeps the latest name per address and shows both. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

LRU::evict_until now returns the removed entries. The server records how long each evicted entry had gone unused, exposes a histogram and median in /v1/stats and on the dashboard, and serves all counters at /metrics in Prometheus text format. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

Copilot

Copilot review overview

🟡 Changes recommended

Concurrency, cache-size enforcement, fail-open behavior, security defaults, and CI test selection have unresolved correctness issues.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 4 · 5

Open (9)

Optional token file allows unauthenticated production uploads · New
Cache keys omit CPU feature requirements · New
Concurrent uploads can collide on the same temporary path · New
Stale temporary files can bypass cache size accounting · New
Binary crate unit tests are skipped in CI · New
Eviction scan reset can lose concurrent upload accounting · New
Corrupt remote blobs abort builds instead of falling back locally · New
Remote cache hits skip cleanup and can exceed the size limit · New
Existence check races with eviction and causes cache miss errors · New

What changed in this PR

Adds a shared HTTP cache server and remote-cache support to the existing Halide cache client.

Changes:

Adds remote lookup/upload support and cross-machine cache keys.
Introduces a cache server with LRU eviction, metrics, dashboard, and service packaging.
Expands concurrent end-to-end tests and CI/release coverage.

File	Description
`lager/src/lru.rs`	Exposes eviction details and skips temporary files.
`lager/src/lib.rs`	Exports new storage APIs.
`lager/src/lager.rs`	Adds raw compressed-blob operations.
`halide-cache/tests/run`	Adds remote and concurrent system tests.
`halide-cache/src/remote.rs`	Implements the HTTP cache client.
`halide-cache/src/main.rs`	Integrates remote caching and key hardening.
`halide-cache/doc/remote-cache-design.md`	Documents architecture and protocol decisions.
`halide-cache/Cargo.toml`	Adds remote-client dependencies.
`halide-cache-server/src/metrics.rs`	Tracks operational cache metrics.
`halide-cache-server/src/main.rs`	Implements the HTTP server and eviction.
`halide-cache-server/src/dashboard.html`	Adds the monitoring dashboard.
`halide-cache-server/contrib/halide-cache-server.service`	Provides systemd deployment configuration.
`halide-cache-server/Cargo.toml`	Defines the server crate.
`Cargo.toml`	Adds the server to the workspace.
`Cargo.lock`	Locks new dependencies.
`.github/workflows/rust.yml`	Expands build and test coverage.
`.github/workflows/release.yml`	Publishes static server binaries.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

RMPR · 2026-09-21T13:28:52Z

+
+[Service]
+ExecStart=/usr/local/bin/halide-cache-server --listen 0.0.0.0:8080 --data-dir /var/lib/halide-cache --size 40GiB
+EnvironmentFile=-/etc/halide-cache-server.env


Done in 82f5d7c: the EnvironmentFile is now mandatory, so the unit refuses to start without the token file.

RMPR · 2026-09-21T13:28:54Z

+        // Generated objects are only valid for the platform they were built on.
+        hasher.update(std::env::consts::OS.as_bytes());
+        hasher.update(&[0u8]);
+        hasher.update(std::env::consts::ARCH.as_bytes());
+        hasher.update(&[0u8]);


Not changed, by design decision during review: the generators are always invoked with an explicit Halide target string, which is part of the hashed command line, so two hosts with different CPU features but the same target produce identical objects. OS and arch were added as a safety net only. If target=host is ever used, the resolved target would have to be passed (e.g. via --builder-id) — noted in the design doc.

RMPR · 2026-09-21T13:28:56Z

+        let mut tmp_name = dest.file_name().unwrap().to_os_string();
+        tmp_name.push(format!("{}-{}", TEMP_SUFFIX, std::process::id()));
+        let tmp = dest.with_file_name(tmp_name);


Done in 82f5d7c: temp names now include a per-process atomic sequence number and are created with create_new. Added a unit test with eight threads storing the same address concurrently.

RMPR · 2026-09-21T13:28:57Z

+                let name = entry.file_name().to_string_lossy();
+                if name.contains(TEMP_SUFFIX) {
+                    // An in-progress write from another process; not part of the cache.
+                    continue;


Done in 82f5d7c: LRU::scan removes temp files older than one hour. Unit test added.

RMPR · 2026-09-21T13:29:01Z

+            t.ages.median_seconds = Some(sorted[sorted.len() / 2]);
+            t.ages.min_seconds = sorted.first().copied();
+        }
+        self.totals.bytes_since_evict.store(0, Ordering::Relaxed);


Done in 82f5d7c: the counter is reset at the start of the eviction pass, before the scan, so bytes arriving during the scan survive and trigger the next pass.

RMPR · 2026-09-21T13:29:03Z

+        hit = cache_hit(
+            &args.generated_object,
+            &args.generated_header,
+            &lager,
+            &object_address,
+            &header_address,
+        )?;


Done in 82f5d7c: extraction failure after a remote fetch is now a warning; both entries are removed and the build proceeds locally. Covered by a new end-to-end scenario that truncates the server's blobs.

RMPR · 2026-09-21T13:29:04Z

+    if hit {
        return Ok(());


Done in 82f5d7c: try_cleaning_up now runs after a remote hit populates the local cache.

RMPR · 2026-09-21T13:29:06Z

+        for kind in [Kind::File, Kind::Dir] {
+            path.set_extension(kind.extension());
+            if path.exists() {
+                let file = File::open(&path)?;
+                file.set_modified(SystemTime::now())?;
+                return Ok((file, kind));
+            }
        }


Done in 82f5d7c: open_raw opens directly and treats NotFound as a miss, including on the mtime touch.

- lager: unique per-write temp names with create_new, so concurrent uploads of one address in a single server process cannot share a temp file - lager: open blobs directly instead of exists()+open, so eviction racing a download reads as a miss rather than an I/O error - lager: remove temp files older than an hour during a scan - server: reset the upload counter before the scan, not after, so uploads landing mid-scan still trigger the next pass - client: a remote blob that fails to extract is discarded with a warning and the build proceeds locally - client: run LRU cleanup after a remote hit populates the local cache - systemd: make the token file mandatory - CI: run the binary crates' unit tests too Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

Client: gather the shared hash inputs in KeyInputs, pair each output's path with its address in Outputs, and express the local/remote/miss flow as a Lookup enum so the fetch, corrupt-blob and cleanup handling no longer nest. The hashed byte stream is unchanged. Server: split main.rs into blobs, eviction, extract and stats modules; use extractors for the blob address and the client identity; keep one Lager in the state instead of reopening it per request; move the eviction age computation into metrics; apply one Delta per event to totals, history and client rows instead of three parallel matches. lager: Lager is Clone; remove() iterates over Kind like open_raw(). tests/run: the helper takes out_dir and base_dir, so the concurrent scenarios reuse it instead of spelling out the command line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

RMPR and others added 13 commits September 20, 2026 13:12

Add remote cache design document

f37b45d

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

CI: check the whole workspace and release static server binaries

645106c

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

doc: update remote cache design to match the implementation

da0dbcc

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

tests: two clients using the server concurrently

328c265

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

CI: build static musl server binaries for both targets

b91e777

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

tests: eight clients racing on the same addresses under eviction

f9eb0fe

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVxsuu7ZCt2cdfn99fuXaA

RMPR requested a balanced review from Copilot September 20, 2026 18:26

Copilot started reviewing on behalf of RMPR September 20, 2026 18:26 View session

Copilot AI reviewed Sep 20, 2026

View reviewed changes

RMPR and others added 2 commits September 21, 2026 15:28

Navigation Menu

Halide server - #7

Halide server#7
RMPR wants to merge 15 commits into
masterfrom
halide-server

RMPR commented Sep 20, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

RMPR Sep 21, 2026

Uh oh!

RMPR Sep 21, 2026

Uh oh!

RMPR Sep 21, 2026

Uh oh!

RMPR Sep 21, 2026

Uh oh!

Uh oh!

RMPR Sep 21, 2026

Uh oh!

RMPR Sep 21, 2026

Uh oh!

RMPR Sep 21, 2026

Uh oh!

RMPR Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

RMPR commented Sep 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants