Conversation
The SGLang OpenAI-compatible API only exposes a server-side whitespace pattern (--constrained-json-whitespace-pattern), so forwarding the field into response_format would be a silent no-op. Raise a clear error instead, matching the llguidance and xgrammar backends. The TGI grammar API has the same limitation, so it now raises too. Closes dottxt-ai#1998
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1998
SGLangTypeAdapterwas silently droppingJsonSchema.whitespace_pattern. The SGLang OpenAI-compatible API doesn't accept a per-request whitespace pattern — it's only configurable at server startup via--constrained-json-whitespace-pattern— so mirroring the vLLM adapter and stuffing the field intoresponse_formatwould have been a silent no-op. I made the adapter raise a clearNotImplementedErrorinstead, which is what thellguidanceandxgrammarbackends already do for the same reason.The TGI adapter had the exact same gap (TGI's grammar API has no whitespace option either), so it now raises too.
SGLangTypeAdapter: raise whenwhitespace_patternis set, with a pointer to the server-side flagTGITypeAdapter: raise whenwhitespace_patternis setwhitespace_patternVerified against the SGLang and TGI servers: neither exposes a per-request whitespace pattern (only the SGLang server flag). Tested with the repo-pinned ruff 0.9.1 (clean) and the sglang/tgi/vllm adapter tests.
Developed with AI assistance and reviewed before submission.