Skip to content

[Test] [B300] kimi_k3: Add matched TP8 DSpark CI coverage - #73

Draft
leejnau wants to merge 5 commits into
sgl-project:mainfrom
leejnau:ci/kimi-k3-b300-e2e
Draft

leejnau wants to merge 5 commits into
sgl-project:mainfrom
leejnau:ci/kimi-k3-b300-e2e

Conversation

@leejnau

@leejnau leejnau commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Add matched Kimi K3 accuracy and performance CI on one host with eight B300 GPUs. Use native SGLang with megakernels disabled as the baseline; keep both arms on identical source and checkpoint revisions.

Modifications

  • Add the B300 manifest entry and two E2E test entry points sharing one sequential stock/plugin server pair.
  • Scope: TP8, five-proposal/q6 DSpark, stock RadixArk draft execution, TRTLLM MLA, DCP off and the existing sm148 opt-in. Check captured B1/B2, eager B3/B4, return to B1 and a 32,768-token passkey input.
  • Record source/checkpoint/dependency identities, launch settings, server logs, quality results and timing samples. Also repair the relocated SGLang context-helper import, retaining a narrowly guarded fallback for the older CPU-CI baseline. No kernel, numerical-threshold or default-activation changes.

Requires both SGLang #51429 (per-model runner workflow) and SGLang #51977 (restored Kimi q6 runtime binding). The existing cross-repository trigger dispatches SGLang main, so both changes must be available there for the normal B300 trigger.

Accuracy Tests

Matched B300 CI run 44815092291 passed: native configuration admission, five framework binding tests plus four subtests, and both E2E tests. Captured/eager transitions and the 32,768-token passkey input passed. GSM8K-200: stock 151/200, plugin 150/200, with zero request errors, within the unchanged 0.005 absolute score-drop threshold. The shared-generated-prefix check passed its unchanged 0.5 logprob bound.

Tested framework 0da13fa8c67001fc88d7d9ed454cd6a2b2f41b70 includes the runtime repair now extracted into #51977; plugin was c5d2baead02d801e6a14b7d81239c02198855190. Current head 70a0c3babe40416375771323b46d295304be88a1 merges upstream e9e01fe, resolving only the manifest/guide conflicts: Kimi code, import tests and the E2E fixture remain byte-identical to the passed run. Upstream's per-model labels and Qwen manifest removal are preserved; B300 instructions now live in AGENTS.md rather than the deleted contribution guide. This is not a new full-pair GPU result. Earlier failed attempts are preserved: relocated context-helper import, unset phase-backend fields, and missing framework q6 binding fields. The first two are corrected here; the third is fixed separately in #51977. After conflict resolution, 16 focused CPU checks plus 72 subtests, runner-selector checks and repository-wide lint pass. Native Linux unit CI is rerunning on the updated head. This is bounded q6/32K-input CI, not new ordinary-decode or 262K qualification.

Speed Tests and Profiling

Three warmed serving repeats, 128 output tokens per request, natural acceptance:

Concurrency / path Stock median tokens/s Plugin median tokens/s Ratio
C1 / captured 236.94 256.60 1.083x
C4 / eager 70.24 212.45 3.024x

Acceptance counts matched in these timing samples (5.12 tokens including bonus per verify step). These are short, bounded end-to-end serving measurements, not a general workload or isolated-kernel speedup claim. The test still fails a candidate slower across all repeats; overlapping ranges are inconclusive.

@leejnau

leejnau commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

Matched eight-B300 CI passed: https://github.com/sgl-project/sglang/actions/runs/44815092291

TP8, five-proposal/q6 DSpark, stock draft execution, DCP off: captured/eager transitions, 32K passkey input, GSM8K-200 and repeated C1/C4 timing all passed. GSM8K was 151/200 stock vs. 150/200 plugin, within the unchanged tolerance; timing medians were 1.083x at C1 and 3.024x at C4, with matched acceptance counts.

This used plugin c5d2bae and repaired framework 0da13fa8c67001fc88d7d9ed454cd6a2b2f41b70. The runtime repair is now a separate SGLang PR: sgl-project/sglang#51977. The normal upstream CI route needs both that repair and the runner workflow in sgl-project/sglang#51429. Description updated; no kernel changes or threshold relaxation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant