fix(types): JSON-quote string-shaped built-in terms inside containers

[feiiiiii5]

Root Cause

Built-in Regex terms from outlines.types handed in directly to a container annotation (e.g. list[types.email], dict[str, types.uuid4]) take the isinstance(ptype, Term) shortcut in python_types_to_terms and come back bare from _ensure_json_quoted. The generated regex matches [a@b.com] and rejects ["a@b.com"] — output that json.loads cannot parse (issue #1962).

Dict keys already work (quote_regex=True); the gap is list items and dict values.

Fix

Invert the quoting rule in _ensure_json_quoted (per the issue's suggested design): identity-match the built-in terms and quote every string-shaped one, while leaving the special groups untouched:

• JSON scalars stay bare (identity): integer, number, boolean, digit.
• Control-character terms stay bare (identity): newline, whitespace, paragraph, sentence — quoting them would look fixed while still failing json.loads (raw control chars / unescaped quotes); escaping them is follow-up work documented in the code.
• Temporals are left to fix(types): JSON-quote date/time/datetime inside DSL containers #1961 (date, time, datetime).
• All other built-in string-shaped terms are quoted: uuid4, ipv4, ipv6, mac_address, semver, slug, hex_color, hex_str, credit_card, char.

Identity matching (not pattern equality) means a user-supplied Regex that happens to share a built-in pattern is left alone.

Test

• test_e2e_string_shaped_builtin_terms_quoted_in_containers — parametrized over the 10 string-shaped terms: list items and dict values now match the quoted spelling and reject the bare spelling; dict keys keep working. (types.email/types.isbn are excluded because their patterns contain ^/$ anchors that cannot be embedded in a container regex at all — a pre-existing limitation independent of quoting.)
• test_e2e_json_scalar_builtin_terms_stay_bare_in_containers — integer/number/digit stay bare.
• test_e2e_control_char_builtin_terms_left_bare_in_containers — the four control-char terms are unchanged.
• test_e2e_user_regex_term_still_bare_in_containers — a user Regex sharing a built-in pattern is untouched.
• Full tests/types suite: 301 passed.

Diff scope

2 files, +115/-0: src/outlines/types/dsl.py, tests/types/test_dsl.py.

AI Disclosure

AI-assisted root-cause analysis, initial draft, and test scaffolding; human review of the identity-matching design and the scalar/control-char carve-outs.

Not PR-related failures

None; the full types suite passes locally. (This PR deliberately avoids the temporal terms to stay disjoint from #1961.)

[github-actions]

📚 Documentation preview: https://dottxt-ai.github.io/outlines/pr-preview/pr-1975/

Preview updates automatically with each commit.

[ErenAta16]

Small mechanical note rather than a review of the logic — this and #1974 overlap in a way that will bite whichever lands second.

Both branches carry the same NoneLiteral block in _ensure_json_quoted, byte for byte:

if isinstance(term, NoneLiteral):
    json_null = Regex("null")
    if quote_regex:
        return Sequence([String('"'), json_null, String('"')])
    return json_null

and both are open against main independently. Comparing the two heads:

6d04c86 (#1974) ... fa6cb81 (#1975)  ->  diverged, ahead 2, behind 1

So this branch started from #1974 and has since drifted — #1974 has one commit this one doesn't. Practical consequences:

• Whoever reviews this one is implicitly reviewing fix(types): render Optional/Union None as JSON null inside DSL containers #1974's change too, but without fix(types): render Optional/Union None as JSON null inside DSL containers #1974's tests, which are on the commit that isn't here.
• If fix(types): render Optional/Union None as JSON null inside DSL containers #1974 merges first, this branch conflicts in _ensure_json_quoted; if this one merges first, fix(types): render Optional/Union None as JSON null inside DSL containers #1974 becomes partly redundant and its remaining commit needs rebasing anyway.

Either marking this one as depending on #1974 and rebasing it on top, or pulling the NoneLiteral block out of here entirely and letting #1974 own it, would make both reviewable on their own.

Worth flagging that dsl.py currently has seven open PRs against it (#1941, #1944, #1946, #1947, #1961, #1974, #1975), and _ensure_json_quoted alone is touched by three of them. Given #1962 is already tracking the shape of the built-in-term rule, it might be worth noting a merge order there so these don't serially invalidate each other's diffs — the design discussion is settled enough that the remaining cost is mostly rebasing.

[feiiiiii5]

Thanks for the note — you're right that the stacking would bite whoever lands second, so I've un-stacked this PR: the branch is now based on main and contains only the string-shaped built-in terms change (1 commit, d344eb8). #1974 keeps the NoneLiteral work; the two can now land in any order.

[ErenAta16]

Thanks for un-stacking it — confirmed on d344eb8: one commit, no NoneLiteral block, +115/-0. The two PRs now carry disjoint changes.

Three things from reading it on its own.

They still collide, but only in the tests

I merged the two branches locally to check whether "any order" holds in practice:

$ git merge p1975      (onto p1974)
Auto-merging src/outlines/types/dsl.py
Auto-merging tests/types/test_dsl.py
CONFLICT (content): Merge conflict in tests/types/test_dsl.py

dsl.py auto-merges cleanly now, which is the part that mattered. What's left is that both PRs append their new tests at the same point in test_dsl.py, so whichever lands second still needs a trivial rebase. Not worth restructuring for — just flagging so it isn't a surprise.

Coverage is complete

I checked the two name tuples against every module-level Regex singleton in types/__init__.py rather than trusting the count in #1962:

Regex singletons in types/__init__.py : 24
covered by the two tuples             : 23
not covered                           : ['string']
named in the PR but not in types      : []

string is the one omission and it's already handled — the term.pattern != types.string.pattern guard sits above both _is_builtin_term calls, so it never reaches them. Nothing falls through to the bare path.

Using identity (term is getattr(types, name, None)) rather than pattern equality is also the right call, and the docstring saying so is worth keeping: a user's own Regex(r"[a-z0-9]+(?:-[a-z0-9]+)*") should not silently start behaving like types.slug.

The email exclusion looks wrong, and it's the case the issue was opened about

The test docstring says:

types.email/types.isbn are excluded from the parameter set because their patterns contain ^/$ anchors that cannot be embedded in a container regex at all

That holds for isbn, which really does carry $ inside its lookaheads. It doesn't hold for email. Stripping character classes before looking for anchors:

email        ^=False  $=False
isbn         ^=False  $=True

Every ^ and $ in the email pattern lives inside a character class — [a-z0-9!#$%&'*+/=?^_{|}~-]` — where they're literal characters, not anchors.

And on your branch it works:

list[types.email]   quoted='["a@b.com"]' -> True    bare='[a@b.com]' -> False
list[types.uuid4]   quoted -> True                   bare -> False
list[types.isbn]    quoted -> False                  bare -> False

email behaves exactly like the types that are in the parameter set. isbn is genuinely broken and the exclusion is right.

This matters more than a missing test row, because list[types.email] producing [a@b.com] is the example in the title of #1962. As it stands the PR fixes the motivating case and then leaves it out of its own regression coverage, so a future change could break it without failing anything. I'd move (types.email, "a@b.com") into the parametrize list and narrow the docstring to isbn alone.

[feiiiiii5]

Addressed in 29ad91a0 — thanks for the precise check on the anchors.

• (types.email, "a@b.com") is now in the parametrize set, so the motivating example from Built-in Regex terms are not JSON-quoted inside containers (list[types.email] generates [a@b.com]) #1962 (list[types.email] must not emit [a@b.com]) is covered by regression tests.
• The docstring now excludes only types.isbn, matching the verified anchor analysis (email’s ^/$ live inside character classes; isbn carries a real top-level $).

Local run: pytest tests/types/test_dsl.py → 75 passed (including the new email row).

[ErenAta16]

Anchors and parametrize set both look right now.

(types.email, "a@b.com") being in the set means the motivating case from #1962 is pinned directly rather than by analogy, and excluding types.isbn with the reason written down beats silently leaving it out.

The test I want to single out is this one:

def test_e2e_user_regex_term_still_bare_in_containers():
    user_regex = Regex(r"[a-z]+@[a-z]+\.[a-z]+")
    list_pattern = to_regex(python_types_to_terms(list[user_regex]))
    assert _re.fullmatch(list_pattern, "[a@b.com]")

That is the safety net for the whole approach. Since Regex.__eq__ is pattern-based, matching built-ins by equality rather than identity would catch a user's own term that happens to share a pattern and start quoting their output. This test fails the moment someone makes that simplification, and it is the failure that would otherwise be reported months later as "outlines corrupts my custom regex". Worth a comment above it saying that is what it is for, because it reads like a redundant negative case.

Same for test_e2e_control_char_builtin_terms_left_bare_in_containers: excluding newline/whitespace/paragraph/sentence rather than quoting them is the right call, and the docstring gives the reason (quoting would look fixed while still failing json.loads), which is the sort of thing that gets "tidied up" without it.

Nothing further from me.

[bharadwaj-pendyala]

I opened #1962, so I ran this branch rather than read it. Checked against current main instead of the branch base, because two type-adding commits landed in between, and three terms fall through the allowlist.

The branch is based on be2cd151 (2026-07-22), which carries 24 built-in Regex terms. main at 7d068478 carries 29. c641fcc8 added iban and bic on 2026-08-05, and 10141976 added e164, latitude and longitude on 2026-08-06. The two tuples in _ensure_json_quoted name 23 of the 29.

Copying only this branch's src/outlines/types/dsl.py onto main at 7d068478, then matching list[term] against both spellings:

term        bare    quoted   generated form                            json.loads
bic         True    False    [DEUTDEFF]                                FAILS
iban        True    False    [DE89370400440532013000]                  FAILS
e164        True    False    [+14155552671]                            FAILS
latitude    True    False    [45.5]                                    parses
longitude   True    False    [-73.6]                                   parses
email       False   True     ["a@b.com"]                               parses
uuid4       False   True     ["123e4567-e89b-42d3-a456-426614174000"]  parses

latitude and longitude are number-shaped, so bare is the right answer for them, though the code reaches it by falling through rather than by classifying them. bic, iban and e164 are string-shaped and come out in exactly the form #1962 is about. [+14155552671] has no reading under which it parses.

Your test file passes unmodified on main, 75 passed, so this is the three new names only and not a regression in what you wrote.

The thing that worries me more than the three names is that the fall-through case is "leave it bare" and it is silent, so every future type PR can quietly reopen #1962. Two did inside ten days. Lifting the two tuples to module level and adding a guard would fail the next one at test time instead:

def test_every_builtin_regex_term_is_classified():
    builtins = {
        name
        for name in dir(types)
        if not name.startswith("_") and isinstance(getattr(types, name), Regex)
    }
    assert not builtins - (BARE_TERMS | QUOTED_TERMS | {"string"})

Happy to push that as a follow-up on top of yours if you would rather keep this PR to the shape it already has.

[feiiiiii5]

Thanks for running the branch against current main — the three fall-throughs are fixed in 052726b1 (on top of a merge with current main).

What changed:

• iban, bic, e164 are now in the quoted allowlist; your three repro rows now emit quoted forms: ["DE89370400440532013000"], ["DEUTDEFF"], ["+14155552671"] all pass json.loads.
• latitude/longitude are now explicitly classified as bare (number-shaped), instead of reaching the bare path by silent fall-through.
• The two allowlist tuples are lifted to module-level BARE_TERMS / QUOTED_TERMS so they can be tested directly.
• Added the guard you proposed, test_every_builtin_regex_term_is_classified: every module-level Regex singleton in outlines.types must be covered by BARE_TERMS | QUOTED_TERMS | {"string"}. A future type PR that adds a singleton without classifying it now fails at test time.
• Parametrize coverage extended: quoted rows for iban/bic/e164, plus a new bare row set for latitude/longitude.

Verified: pytest tests/types → 368 passed; ruff check (v0.9.1, pinned) clean on the changed files.

You offered to push the guard as a follow-up — no need, it's in here. Happy to split it out if you'd rather review it separately.

[feiiiiii5]

Closing as part of a contribution-portfolio consolidation: my effort is now focused on a single project (failroute, a static analyzer for silent-failure anti-patterns) and its upstream follow-through. Happy to revisit if there is interest.

[feiiiiii5]

Reopening — the 08-28 close was a mistake on my side. Same family as #1974.

The three fall-throughs @bharadwaj-pendyala hit when he ran the branch against current main are fixed in 052726b1, on top of a merge with current main.

Sorry for the noise.

[ErenAta16]

Reviewing this rather than the newer branch on the same problem, since this one
was first and the classification here is the part that needs a decision.

The structure is right. Matching built-ins by identity rather than by pattern is
the detail I would have got wrong, and the comment says why: a user's own Regex
that happens to share a pattern must not inherit built-in treatment. Splitting the
vocabulary into two named tuples also makes the review a reading of two lists
instead of a reading of control flow, which is the right shape for something that
has to be audited term by term.

Two of the thirteen entries in BARE_TERMS look misclassified to me, and one of
them is not a judgement call. Measured on the two forms this PR chooses between:

list[whitespace]  bare    '[ ]'     -> json.loads gives []        value is gone
list[whitespace]  quoted  '[" "]'   -> json.loads gives [' ']     correct

list[newline]     bare    '[\n]'    -> json.loads gives []        value is gone
list[newline]     quoted  '["\n"]'  -> JSONDecodeError: Invalid control character

whitespace is the clear one. types.whitespace is Regex(r"\s"), a single
character, and quoting it produces exactly the value the schema declares. Left
bare it does not merely look wrong, it parses to an empty list, so a caller gets
a well-formed result with the element silently missing. That belongs in
QUOTED_TERMS, and the argument for it is the same one the PR already makes for
email and the rest: the term is string-shaped, not number-shaped.

newline is genuinely awkward and I am not asking you to solve it here. Neither
classification is correct, because a literal newline is not legal inside a JSON
string and the real fix is escaping rather than quoting. What the measurement
does say is that bare gives a silent wrong answer and quoted gives a loud one, so
if this has to ship as a binary choice, quoted is the safer half. Worth a line in
the description saying which you chose and why, so the next person does not read
newline sitting next to integer and assume it was audited as a scalar.

The other eleven bare entries hold up. integer, number, boolean, digit,
latitude and longitude are all number-shaped and correct bare, and the comment
change on latitude/longitude in types/__init__.py is a real improvement
because it replaces a stale "currently generated unquoted (#1962)" note with the
reason the current behaviour is intended.

sentence and paragraph I would leave where they are. They fail loudly either
way, so the classification does not change what a caller sees, and moving them
would only add churn to a diff that is already the audit surface.

[ErenAta16]

Completing my review from earlier today, because I gave you one example when I
should have given you the list, and the missing part changes what this branch
closes.

I said whitespace was misclassified as bare. That is right, but it is not one
term, it is seven. I checked out this branch and #2015 and called
_ensure_json_quoted on every built-in Regex in outlines.types, rather than
reading the tuples:

term                                          main   #1975   #2015
integer number boolean digit lat long         bare   bare    bare
whitespace newline sentence paragraph         bare   bare    quoted
date time datetime                            bare   bare    quoted
email uuid4 ipv4 isbn semver slug hex e164    bare   quoted  quoted

The temporal row is the one that matters most, because it has its own report.
#1960 is open specifically about date/time/datetime being left unquoted in
containers, and on this branch they still are:

[2024-01-01]     -> JSONDecodeError
["2024-01-01"]   -> ['2024-01-01']
[12:30:00]       -> JSONDecodeError
["12:30:00"]     -> ['12:30:00']

So if this lands as the fix for the string-shaped terms, #1960 stays open and
the eight terms it does quote can read as the whole job being done. That is worth
saying out loud in the description at minimum.

None of this is a criticism of the classification work itself. The identity match
is the right mechanism and I still think so. It is the polarity that decided the
outcome: an allowlist has to name every string-shaped term, and the seven that
were missed are exactly what happens when one is not named. #2015 lists the JSON
scalars instead and quotes everything else, and its docstring makes the same
argument in advance, which the measurement above now backs up rather than just
asserting.

Two ways forward and both are fine by me. Move the seven from BARE_TERMS to
QUOTED_TERMS here, which also closes #1960 and makes the two branches agree on
all fifteen. Or treat #2015 as the one for this row, in which case the useful
thing from here is the latitude/longitude reasoning, which both branches got
to independently and which #2015 flags as a separate bug because of the leading
+.

@bharadwaj-pendyala this is the comparison for the pair, so you should see it too.

[ErenAta16]

Withdrawing part of what I said this morning, because I overstated it.

I called the whitespace classification a misclassification rather than a
judgement call. That was too strong. I had not read
test_e2e_control_char_builtin_terms_left_bare_in_containers, which states the
reasoning outright: quoting a control-character term would look fixed while still
failing json.loads. Measured across what \s actually matches:

              bare        quoted
space         []          [' ']          correct
tab           []          JSONDecodeError
newline       []          JSONDecodeError
CR            []          JSONDecodeError

So quoting rescues one of the four and makes the other three loud, against bare
being silently empty in all four. I still think quoted is the better half of that
trade, since a silent empty list is the worse failure, but it is a trade with a
stated reason on your side and I presented it as if there were nothing to weigh.
That was unfair to the branch.

What I said about the temporal terms does not change, and it is the part that
matters:

              bare              quoted
2024-01-01    JSONDecodeError   ['2024-01-01']
12:30:00      JSONDecodeError   ['12:30:00']

No control characters are involved, there is no trade-off, bare is a hard error
and quoted is simply correct. date, time and datetime sit in BARE_TERMS
alongside integer and number without the rationale the control-character group
has, and #1960 is open specifically about them. That is the one I would still ask
you to move.

[feiiiiii5]

Thanks @ErenAta16 — running _ensure_json_quoted across every built-in term settles it, and I'll take your option (b).

@bharadwaj-pendyala (who reported both #1960 and #1962) has #2015 open. Its scalar-denylist polarity quotes every string-shaped term by default, so it covers date/time/datetime and the seven terms this branch's allowlist missed, while closing #1962 and superseding #1961. That is the more complete fix for this family, so I won't expand #1975 to race it. The latitude/longitude leading-+ point that both branches reached independently is flagged as its own bug on #2015 too.

I'll leave it to the maintainers whether to close this as superseded by #2015 or land them in whichever order. No further changes from my side here.

[feiiiiii5]

Closing in favor of #2015: @bharadwaj-pendyala (reporter of #1960 and #1962) covers this whole family with a scalar-denylist that quotes every string-shaped term including date/time/datetime, closing #1962 and superseding #1961. Thanks @ErenAta16 for the term-by-term measurement that made the call clear. Branch preserved; happy to reopen if #2015 stalls.