Repository navigation
Conversation
Contributor
Author
|
Matched eight-B300 CI passed: https://github.com/sgl-project/sglang/actions/runs/44815092291 TP8, five-proposal/q6 DSpark, stock draft execution, DCP off: captured/eager transitions, 32K passkey input, GSM8K-200 and repeated C1/C4 timing all passed. GSM8K was 151/200 stock vs. 150/200 plugin, within the unchanged tolerance; timing medians were 1.083x at C1 and 3.024x at C4, with matched acceptance counts. This used plugin c5d2bae and repaired framework 0da13fa8c67001fc88d7d9ed454cd6a2b2f41b70. The runtime repair is now a separate SGLang PR: sgl-project/sglang#51977. The normal upstream CI route needs both that repair and the runner workflow in sgl-project/sglang#51429. Description updated; no kernel changes or threshold relaxation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Add matched Kimi K3 accuracy and performance CI on one host with eight B300 GPUs. Use native SGLang with megakernels disabled as the baseline; keep both arms on identical source and checkpoint revisions.
Modifications
sm148opt-in. Check captured B1/B2, eager B3/B4, return to B1 and a 32,768-token passkey input.Requires both SGLang #51429 (per-model runner workflow) and SGLang #51977 (restored Kimi q6 runtime binding). The existing cross-repository trigger dispatches SGLang
main, so both changes must be available there for the normal B300 trigger.Accuracy Tests
Matched B300 CI run 44815092291 passed: native configuration admission, five framework binding tests plus four subtests, and both E2E tests. Captured/eager transitions and the 32,768-token passkey input passed. GSM8K-200: stock 151/200, plugin 150/200, with zero request errors, within the unchanged 0.005 absolute score-drop threshold. The shared-generated-prefix check passed its unchanged 0.5 logprob bound.
Tested framework
0da13fa8c67001fc88d7d9ed454cd6a2b2f41b70includes the runtime repair now extracted into #51977; plugin wasc5d2baead02d801e6a14b7d81239c02198855190. Current head70a0c3babe40416375771323b46d295304be88a1merges upstreame9e01fe, resolving only the manifest/guide conflicts: Kimi code, import tests and the E2E fixture remain byte-identical to the passed run. Upstream's per-model labels and Qwen manifest removal are preserved; B300 instructions now live inAGENTS.mdrather than the deleted contribution guide. This is not a new full-pair GPU result. Earlier failed attempts are preserved: relocated context-helper import, unset phase-backend fields, and missing framework q6 binding fields. The first two are corrected here; the third is fixed separately in #51977. After conflict resolution, 16 focused CPU checks plus 72 subtests, runner-selector checks and repository-wide lint pass. Native Linux unit CI is rerunning on the updated head. This is bounded q6/32K-input CI, not new ordinary-decode or 262K qualification.Speed Tests and Profiling
Three warmed serving repeats, 128 output tokens per request, natural acceptance:
Acceptance counts matched in these timing samples (5.12 tokens including bonus per verify step). These are short, bounded end-to-end serving measurements, not a general workload or isolated-kernel speedup claim. The test still fails a candidate slower across all repeats; overlapping ranges are inconclusive.