Skip to content

Fix JSON decoding of nested dictionary environment values - #1109

Merged
hramezani merged 2 commits into
pydantic:mainfrom
Hugohong258:fix-nested-dict-env-decoding
Oct 28, 2026
Merged

hramezani merged 2 commits into
pydantic:mainfrom
Hugohong258:fix-nested-dict-env-decoding

Conversation

@Hugohong258

Copy link
Copy Markdown
Contributor

For dict[str, list[dict[str, str]]], an environment value such as APP_VALUES__X='[{"k":"v"}]' reaches validation as a string. next_field() descends into the dictionary's type arguments before recognizing the dictionary itself, losing the value's container type. Handle the outer dictionary first so the value is decoded at the correct nesting level. Fixes #1107.

Three new cases fail on the base and one control passes. With the change, the settings module passes 515 tests with 7 skips on Windows/Python 3.12.3; settings and nested-secret tests pass 577 with 1 skip on Linux/Python 3.14.4. Repository-wide Ruff lint/format and mypy (22 library files) pass using the lockfile tools. The broader Windows unit run excluding documentation examples has 1,276 passes, 11 skips and 6 failures while creating symlinks (WinError 1314); these cases pass in the Linux selection. Documentation examples and the full OS/interpreter matrix were not run.

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 28, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 74c01ae

Macroscope's review found this PR approvable — Preserves dictionary value types so nested environment values receive JSON decoding at the intended level. Focused regression coverage and unchanged source-access behavior keep the change's risk limited.

Notes:

  • No code objects were reviewed. Approvability was decided on eligibility alone.

You can add or adjust custom eligibility rules. Learn more.

@flyoung588

flyoung588 commented Oct 28, 2026 •

Copy link
Copy Markdown

AI-assisted supplemental regression evidence: this also fixes model-valued dictionary keys that coincide with the value model's field names or validation aliases, beyond the nested-container cases currently in the PR.

For example, with values: dict[str, Value], Value containing name: str and count: int, and env_nested_delimiter='__', PROBE_VALUES__NAME='{"name":"ok","count":7}' fails on the base: next_field() mistakenly selects the value model's name field before consuming the dictionary key, so the whole JSON object remains a string. A key OTHER passes. COUNT fails too; an aliased model adds the same failure for its validation_alias='label' key.

I ran seven cases against actual base 7e2dacb6db07b41abb0c80d0608e46e197265554 and PR head 36f3d9f481df0310ee75370c06017d8825ed5447 on macOS / CPython 3.11.12: base 5 failed / 2 passed; this PR 7 passed. The two non-colliding-key cases are controls. The existing change is sufficient; no extra production patch is proposed.

Pinned runnable probe, raw outputs and exact dependencies (evidence commit b4a2f29d3f00d0d23e8fcb04b578cd49ee20d95a). Run from that checkout with PYTHONPATH=. python -m pytest experiments/model-dict-key/probe_model_keys.py -v.

This gives an additional regression shape to consider for the PR tests. Focused evidence only, not a full review or platform-matrix claim; no duplicate fix PR.

Update: I converted these into native tests in tests-only collaboration PR to your existing branch, code commit 8178898fe7fb07232cef52d05ff9080c1f6c27af. Full tests/test_settings.py: 528 passed, 1 skipped; Ruff check/format pass. Raw outputs. This offers the cases for your PR; it does not indicate that you have run or accepted them.

Move the regression test next to the other nested dict env tests and
cover more shapes from pydantic#1107: Annotated value types, Optional dicts,
dicts inside models (and models inside dicts), and dict keys that
match a field name of the dict's value model.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRaoYMkPxtZ35KHCmqCYHf
@macroscopeapp
macroscopeapp Bot dismissed their stale review October 28, 2026 09:53

Dismissing prior approval to re-evaluate 74c01ae

@hramezani
hramezani enabled auto-merge (squash) October 28, 2026 09:55
@hramezani

Copy link
Copy Markdown
Contributor

Thanks @Hugohong258

@hramezani
hramezani merged commit 9e02de0 into pydantic:main Oct 28, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nested env var is not JSON-decoded when a dict field's value type contains another dict

3 participants