Skip to content

Upgrade llama.cpp from b11512 to b11529; nvcc through sccache opt-in; docs cleanup - #494

Merged
bernardladenthin merged 7 commits into
mainfrom
claude/hopeful-pascal-9jlbqb
Oct 9, 2026
Merged

bernardladenthin merged 7 commits into
mainfrom
claude/hopeful-pascal-9jlbqb

Conversation

@bernardladenthin

Copy link
Copy Markdown
Owner

Summary

  • Upgrades the pinned llama.cpp from b11512 to b11529 in 5 reviewed steps, each ending at a tag (b11516, b11517, b11518, b11521, b11529; chunked by diff size per docs/upgrade/llama-cpp-version-bump.md, with the one 530 KiB step b11518→b11521 taken whole because no tag sits between them). All eleven patches apply unchanged, verified in order against each pristine tag with git apply and, for b11529, through the real FetchContent configure; every drop-check still finds its defect (no kolibri upstream, no split_sum guard, no common_log sink, no stop_server, the four _mm_prefetch calls still uncast, …). Two history rows per step in docs/history/llama-cpp-breaking-changes.md.
  • One behaviour change from upstream, handled: #30159 moves the server's default port from 8080 to 9931. NativeServer forwards its argv to llama_server, so an argv without --port now binds 9931 in classic and attach mode; NativeServer.DEFAULT_PORT (what getPort() reports in that case) follows, with NativeServerSmokeTest and a CHANGELOG entry. OpenAiCompatServer keeps its own DEFAULT_PORT = 8080 on purpose: the Java transport has no upstream coupling and the constant is a public API of this project. Every README example passes --port explicitly.
  • Notable in the range (no project source change needed): CUDA top-k selection reworked (#28713) and MMQ config helpers take the src1 precision (#30168); SYCL Q5_K reorder-layout MMVQ + fused GLU (#29375); Metal flash-attention kernels for the 128/96 head-size pair (#30209); Vulkan rms_norm workgroup-count overflow fix (#30145); ggml_acc out-of-bounds write with a large offset fixed (#30135); meta backend handles host views under --split-mode tensor (#30217); the WebUI gains a models manager (#29583, auto-followed by build-webui; package.json adds vaul-svelte, scripts/ui-assets.cmake unchanged). Request fields, bounds and response keys of tools/server are untouched in the range (server-schema.cpp / server-task.cpp did not change).
  • build.sh: nvcc through sccache is opt-in now (SCCACHE_WRAP_NVCC=true), off by default. Every Linux CUDA job log examined since the CUDA 13.4 bump (runs 37643964068, 37680715063, 37831180985, 37925157175) shows sccache 0.18.0 failing on the first .cu TUs ten seconds into the build (Missing "cubin" file output, Compiler killed by signal 126) and the retry rebuilding everything without any launcher, gcc objects included, for 47–80 min. The nvcc cache never hit on 13.4, and the retry threw the gcc cache away with it. By default sccache now wraps the C/C++ compilers only; ggml adds no launcher of its own in that state (its GGML_CCACHE block fires only when neither C nor CXX launcher is set). Expected effect on the CUDA job: gcc objects served from the cache, nvcc at the cold cost (~50 min) with no wasted first attempt — this PR's run is the first measurement. CLAUDE.md, TODO.md and the CHANGELOG record it and how to re-test the opt-in with a newer sccache.
  • Docs cleanup: CLAUDE.md's 0017 row no longer names the deleted .github/workflows/probe-msvc-prefetch.yml.

Test plan

  • Patches: git apply in order on pristine b11516, b11517, b11518, b11521, b11529 (eleven of eleven each); check-patches.py: 11 patches, 98 hunks, 0 problems
  • Fresh configure at b11529 through FetchContent + full build; ctest: 603/603 passed
  • Java: mvn test on the llama module: 1869 run, 0 failures, 280 skipped (the model-gated classes self-skip here); NativeServerSmokeTest with the new default
  • javadoc:jar, compile spotbugs:check, spotless:check
  • check-natives.py (0 disagreements), check-release-gate.py, check-run-scripts.py, check-shared-files.py (0 changed here alone), buildcheck unit tests (112), bash -n .github/build.sh
  • CI is green on this branch — the CUDA job's duration and its sccache stats table are the measurement for the build.sh change
  • Docs / CHANGELOG updated where applicable

Related issues / PRs

Checklist

  • I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  • My commits follow Conventional Commits
  • No security-sensitive changes (if there are, I have notified the maintainer privately per SECURITY.md)

🤖 Generated with Claude Code

https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2


Generated by Claude Code

claude added 7 commits October 9, 2026 16:16
`.github/workflows/probe-msvc-prefetch.yml` was a one-off that printed the
`_mm_prefetch` declaration of every MSVC toolset on the Windows image and was
removed once it had answered; the row still pointed at it as if it existed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
All eleven patches apply unchanged (verified in order against the pristine
tag); see docs/history/llama-cpp-breaking-changes.md for the range.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
All eleven patches apply unchanged (verified in order against the pristine
tag); see docs/history/llama-cpp-breaking-changes.md for the range.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
All eleven patches apply unchanged (verified in order against the pristine
tag); see docs/history/llama-cpp-breaking-changes.md for the range.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
All eleven patches apply unchanged (verified in order against the pristine
tag); see docs/history/llama-cpp-breaking-changes.md for the range.

Upstream #30159 moves the server's default port from 8080 to 9931, which
NativeServer inherits whenever the argv carries no --port (classic and attach
mode alike): NativeServer.DEFAULT_PORT, what getPort() reports in that case,
follows it, with the smoke test and a CHANGELOG entry. OpenAiCompatServer
keeps its own default of 8080 -- the Java transport has no upstream coupling
and the constant is a public API of this project.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
All eleven patches apply unchanged (verified in order against the pristine
tag); see docs/history/llama-cpp-breaking-changes.md for the range.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
…NVCC opts in)

Since the CUDA 13.4 bump every Linux CUDA job log examined (runs 37643964068,
37680715063, 37831180985, 37925157175) shows sccache 0.18.0 failing on the
first .cu translation units ten seconds into the build -- `Missing "cubin"
file output`, `Compiler killed by signal 126` -- and build.sh's retry then
rebuilding everything without any launcher, gcc objects included, for 47 to
80 minutes depending on the runner. The nvcc cache never produced a hit on
13.4, and the retry threw the gcc cache away with it.

Wrapping nvcc is now opt-in (SCCACHE_WRAP_NVCC=true); by default sccache
wraps the C/C++ compilers only and nvcc runs directly from the start. ggml
does not add its own launcher in that state (its GGML_CCACHE block fires only
when neither C nor CXX launcher is set). CLAUDE.md, TODO.md and the CHANGELOG
record the measurements and how to re-test the opt-in with a newer sccache.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
@sonarqubecloud

sonarqubecloud Bot commented Oct 9, 2026

Copy link
Copy Markdown

@bernardladenthin
bernardladenthin merged commit 01f6cc2 into main Oct 9, 2026
67 of 71 checks passed
@bernardladenthin
bernardladenthin deleted the claude/hopeful-pascal-9jlbqb branch October 9, 2026 18:33

This branch had an error being deployed

1 failed and 1 active deployments
startgate — 94847e78 Deployed Oct 9, 2026 by bernardladenthin via Start gate (abort window) #1143
maven-central — 94847e78 Deployed Oct 9, 2026 by bernardladenthin via Verify GPG signing key (no secrets printed) #1143
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants