Add native kron (Kronecker product) affine atom#101
Draft
Transurgeon wants to merge 4 commits into
Draft
Conversation
cvxpy's kron(A, B) always has one variable-free operand, so every output entry depends on a single child entry: Z[OUT] = coeff[OUT] * child[child_row[OUT]]. The output Jacobian is therefore the child Jacobian's rows gathered (with repetition) and scaled by the variable-free operand -- no coefficient matrix, no matmul, no CSC conversion; O(nnz(result)). child_row[] and coeff_idx[] depend only on the operand shapes and are precomputed once in new_kron. Handles kron(param/const, var) and kron(var, param/const), parametric or constant, with column-major (Fortran) flattening, and re-evaluates the variable-free operand each solve. forward, Jacobian and the affine Hessian backprop are all scaled gathers. Adds forward/Jacobian/wsum_hess unit tests (both forms, scalar operand, and numerical Jacobian/Hessian checks on a composite arg); all_tests now 405. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apply clang-format and tidy the native kron atom: factor parameter refresh into a helper, null freed pointers, and trim doc duplicated by the kron_expr definition in subexpr.h. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…matmul block_left_multiply_fill_sparsity tested every row of A against each (column, block) via has_overlap — O(n_vars x n_blocks x A_rows) at Jacobian-init time. Gather instead: build a CSC view of A once, walk only the columns hit by the block's child entries, dedup rows with a generation-stamp workspace and sort ascending. Output is byte-identical (same predicate, same order), so fill_values is untouched. csr_csc_matmul_alloc had the same full row-times-column scan; same treatment via a one-time CSR view of B. Tests: dedup and interleaved-order cases that fail without the stamp or the sort, plus randomized cross-checks against the original scan kept as a reference implementation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
new_kron materialized every output row, including the structurally-zero blocks of a sparse constant operand — kron(I_p, X) built p^2 blocks of Jacobian rows with p(p-1) of them zero. Replace it with new_left_kron / new_right_kron taking the constant operand's active (nonzero) block indices, column-major; inactive output rows keep child_row == -1 and contribute a zero value and an empty Jacobian row. cvxpy passes the constant's nonzeros (all blocks for an updatable parameter, whose zeros are not permanent). Constructors assert active block indices are in range. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cvxpy's kron(A, B) always has one variable-free operand, so every output entry depends on a single child entry: Z[OUT] = coeff[OUT] * child[child_row[OUT]]. The output Jacobian is therefore the child Jacobian's rows gathered (with repetition) and scaled by the variable-free operand -- no coefficient matrix, no matmul, no CSC conversion; O(nnz(result)). child_row[] and coeff_idx[] depend only on the operand shapes and are precomputed once in new_kron.
Handles kron(param/const, var) and kron(var, param/const), parametric or constant, with column-major (Fortran) flattening, and re-evaluates the variable-free operand each solve. forward, Jacobian and the affine Hessian backprop are all scaled gathers.
Adds forward/Jacobian/wsum_hess unit tests (both forms, scalar operand, and numerical Jacobian/Hessian checks on a composite arg); all_tests now 405.