Skip to content

[Bug] Store limit dedup emits duplicates on id-only scans because ProjectionStage mutates the stored column #3407

Closed

Description

@vaidikgarg-bit

In the review of #3404, @bitflicker64 found that MapLimitIterator keeps each emitted RocksDBSession.BackendColumn in its dedup set and passes the same object downstream. On an id-only PRIMARY_SCAN (null_property = true, no property list, condition, order-by or aggregation) with dedup_option = LIMIT_DEDUP, ProjectionStage.handle() then sets column.value = new byte[0] in place (ProjectionStage.java, the BACKEND_COLUMN branch). The stored key changes after insertion, so it no longer equals a later byte-equal row, and the duplicate is emitted again. Their probe gave 100 rows without the reset and 200 with it.

Only a gRPC client that sets dedup_option reaches this today, since StoreQueryParam fixes it to NONE.

Proposed fix: have ProjectionStage replace the column instead of mutating it (result.setColumn(BackendColumn.of(column.name, new byte[0]))), plus a regression test that runs each emitted row through the projection before reading the next. I'd like to work on this.

Activity

  1. added a commit that references this issue on Oct 27, 2026
    e497efd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions