Repository navigation
fix: stop rejecting spatial queries while a transaction commits - #37
Merged
Emskiqq merged 2 commits intoNov 9, 2026
Merged
Conversation
Spatial queries failed with a "pending GraphDB transaction ... force-reindex" error whenever they overlapped another transaction's commit. The plugin marked the index unreadable as soon as it published its Lucene commit and only cleared that once GraphDB finished, so under a steady write load most queries failed even though nothing was wrong. Queries now read the published Lucene commit during that window, the same as before 4.0. The pending-transaction marker is still written before the Lucene commit, so a crash mid-commit still keeps queries blocked after restart until a force reindex. The recovery error is now reserved for outcomes that really are unresolved, and reads "unresolved GraphDB transaction outcome". Any failure during rollback, including an Error, sets it, rewrites the marker and clears the transaction state; before, a later unrelated transaction could quietly clear it. The read gate flags are volatile so query threads see these changes. The new tests include a query from a second connection while the writer is paused after its Lucene commit, plus the abort, failed-commit, failed-rollback and force-reindex recovery paths.
While a transaction's Lucene commit is published but GraphDB hasn't finished the transaction, Lucene can return entities that the transaction itself created. A query that returned those matches directly failed with "Cannot retrieve Value with ID of N", because its connection can't resolve them yet. The relation iterator now skips candidates whose entity ID the query's connection can't resolve, logging them at debug. They aren't in the RDF that query reads, so leaving them out keeps the result closer to the committed state, and they show up in later queries once the transaction completes. The tests cover both binding directions on both the exact-evaluation and envelope-proven disjoint paths, plus a plain spatial query from a second connection while a commit is paused.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #34.
Problem
Since 4.0, spatial queries can fail when they run while another transaction is committing. The plugin marks the index unreadable after its Lucene commit and only makes it readable again once GraphDB finishes the transaction. With writes happening continuously, most queries end up failing with an error asking for a force reindex, even though nothing has gone wrong and no rebuild is needed.
Queries returning spatial matches directly can also fail with
Cannot retrieve Value with ID of N. Lucene can return entities that the committing transaction just created, before the querying connection can see them. This also happens in 3.x.Fix
Queries now keep reading the published Lucene commit while GraphDB finishes the transaction, as they did before 4.0. If a match refers to an entity the querying connection can't resolve yet, we skip it for that query. It can appear in later queries once it becomes visible.
This fixes the query failures reported in the issue, but queries don't wait for the spatial index and RDF data to become aligned. They can still briefly see different states during a commit: geometry changes on existing entities may appear before their RDF statements are visible, and deleted or changed geometries may briefly be missing.
If the RDF transaction aborts normally, the plugin restores the previous Lucene commit, so no reindex is needed.
The transaction marker is still written before the Lucene commit. If a process crash leaves that marker behind, the plugin cannot establish whether the RDF transaction committed, so it requires a force reindex after restart.
If rollback itself fails, including by throwing an
Error, the index also stays blocked until a force reindex. The recovery error now says "unresolved GraphDB transaction outcome".Tests
The regression test runs the issue's batch-check query while a writer is paused after its Lucene commit. It sees no partial batches, and the final count includes all committed batches. It also covers queries that return spatial matches directly.
Additional tests cover aborts, commit and rollback failures.