Skip to content

docs: explain gradients with respect to operator parameters - #259

Closed
AHMETHAKANBEZIR1 wants to merge 1 commit into
patrick-kidger:mainfrom
AHMETHAKANBEZIR1:docs/operator-gradient-pytrees
Closed

AHMETHAKANBEZIR1 wants to merge 1 commit into
patrick-kidger:mainfrom
AHMETHAKANBEZIR1:docs/operator-gradient-pytrees

Conversation

@AHMETHAKANBEZIR1

Copy link
Copy Markdown

Gradients with respect to a composite or scaled linear operator contain derivatives of its PyTree array leaves. Calling .as_matrix() on that returned PyTree can misleadingly interpret those derivatives as another operator, rather than derivatives with respect to dense matrix entries.

Add an FAQ explanation and a scaled-identity example: the scale derivative is 3, while differentiating the trace with respect to a 3-by-3 dense matrix gives the identity. Explain that the same distinction applies to composite operators and equinox.filter_grad.

Closes #144, where the maintainer invited a documentation PR. Only docs/faq.md changes; no runtime behavior changes.

Validation (Windows CPU, Python 3.12.14, JAX 0.11.2):

  • Executed both new Python blocks directly from the Markdown and checked scalar and dense-matrix derivatives with NumPy assertions.
  • Checked the scalar derivative with equinox.filter_grad as well.
  • mkdocs build --strict passed without warnings; checked the generated FAQ heading and code blocks.
  • Applicable pre-commit hook (mixed line endings) and git diff --check passed. Python-only hooks were skipped for this Markdown-only change.

The runtime suite was not rerun for this documentation-only change. Linux and GPU checks were not run locally.

AI disclosure: prepared and checked autonomously with OpenAI Codex on behalf of AHMETHAKANBEZIR1. No independent human review is claimed.

Co-authored-by: OpenAI Codex <noreply@openai.com>
@jpbrodrick89

Copy link
Copy Markdown
Collaborator

This is pretty nice, thanks! I am generally happy to accept purely AI generated PR's for code, but for documentation I would appreciate a human review before passing it over. The example is good and fairly well explained but the introduction is a bit too LLM-y. Can you give it a human pass over before I review, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gradients wrt LinearOperators on scalar functions not matching expectations

2 participants