Repository navigation
fix: share one GeoSPARQL plugin runtime across repositories - #32
Merged
Merged
Conversation
GraphDB loads the plugin through a separate class loader per repository and closes it on deletion. Bundled Apache SIS and Derby then retained every deleted repository's runtime, and only the first repository in a process could open the SIS_DATA EPSG database (Derby XSDB6). A thin GraphDB entry point now forwards to a GeoSparqlPlugin loaded by one never-closed runtime class loader per plugin installation, published in the platform MBean server. Add archive lifecycle tests for deleted, restarted, re-enabled, failed and EPSG-indexing repositories, expected-failure tests for the SIS, Derby, JDK and GraphDB retention limitations, and an SDK interface parity test for the entry point.
The crash-recovery test read the child process output only after waitFor returned, so a child that filled the pipe buffer blocked on write while the test waited for it to exit. The abort-restoration boundary logs about 9 KB of transaction-failure stack traces, which exceeds the reduced pipe buffer the kernel assigns once the per-user pipe limit is reached. Write the output to a file, forcibly stop a child that does not terminate, and include its output in the timeout failure message.
…ions The repository-availability scenario now fails if the repository is not deleted. A new archive lifecycle scenario checks that entry points from one plugin directory share a runtime and that a separately installed directory loads its own.
Four repositories with different projected CRSes build their indexes at the same time and then query concurrently, so first-time and repeated EPSG lookups all go through the shared SIS and Derby runtime.
…tion A timed-out lifecycle process is now stopped and its log tail is included in the failure before the work directory is removed. A new scenario loads several entry points from one plugin directory at the same instant and checks that they create and share one published runtime.
If GraphDB's plugin class loader does not expose the plugin jars, the entry point loads the plugin in each repository's class loader. Log a warning at initialization in that case, since repositories then cannot share the SIS_DATA EPSG database and deleted repositories keep their runtime.
Move publication of the runtime class loader into a method that takes the MBean server, and test with a stub server that reports another repository's registration that the created class loader is closed and the published one is used.
…recovery Four repositories created through the repository manager at the same instant must share the one published runtime, with one set of SIS threads, while each indexes its own projected CRS. The repository whose calls created the runtime and opened the EPSG database is deleted while three others run EPSG queries on worker threads; those queries keep succeeding, the deleted plugin instance is collected, and the thread census is unchanged. A repository created with a deleted repository's ID indexes and queries EPSG data again, which needs the shared Lucene lock registry to have released the deleted index's lock. After failed initializations, a repository created through the manager evaluates filter functions and indexes EPSG:3006 data. The EPSG query loop, the index-and-query steps, and the failing initializations move into helpers shared with the existing scenarios, whose behaviour is unchanged.
A per-repository runtime reintroduces the per-repository memory retention and loses shared SIS_DATA EPSG access, so report it at error level where monitoring alerts.
Java 25 class loaders and threads capture no access-control context, so the lifecycle tests now require the class loader of the repository whose calls created the runtime to be collected. The class loader of the repository that first opens the EPSG database is still kept: Derby creates the static exceptions exceptionClose and NO_MEM of its embedded connection class when that class initializes, and their captured stack traces hold the entry point class. An expected-failure test covers that Derby limitation in place of the JDK one and fails once that class initializer creates no exception. The stopped-SIS control no longer needs a warm-up entry point, because the class loader whose calls created the runtime is now collected. Verified on OpenJDK 25.0.1 with GraphDB 11.0, because the GraphDB 12 artifacts need credentials for maven.ontotext.com. From a git archive copy of this commit with src/test/java/com/ontotext/trree/geosparql/TestGeoSparqlPluginWithRDFStarData.java removed, which needs RDF4J 6: JAVA_HOME=<jdk-25> mvn -o -Dgraphdb.version=11.0-spring-6-TR18 -Dmaven.compiler.release=21 -Dmaven.test.additionalClasspath=<path to trove4j-2.0.2.jar> -Dtest=NoSuchTest -Dsurefire.failIfNoSpecifiedTests=false package GraphDbPluginArchiveIT passes 18 of 18. The retention chain comes from a heap dump of the indexed-epsg-repositories scenario on that JDK: EmbedConnection.exceptionClose, its Throwable backtrace, the GeoSparqlPluginFacade class, the repository class loader; NO_MEM holds the class the same way. On JDK 25 javap shows no URLClassLoader.acc and no Thread.inheritedAccessControlContext, both present on JDK 21.0.11.
The pom compiles for release 25, which the Java 21 toolchain cannot build.
edmondchuc
force-pushed
the
geosparql-mem-fix
branch
from
October 22, 2026 03:16
4d826a2 to
ab2aa2a
Compare
Contributor
Author
|
Note that CI is failing because it cannot resolve the transitive dependency |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #31.
Problem
GraphDB loads a separate copy of each plugin for every repository and discards it when the repository is deleted. The GeoSPARQL plugin bundles Apache SIS, which is designed to run once per process: it starts daemon threads and registers a JVM shutdown hook that keep its copy of the plugin in memory. Every repository create/delete cycle therefore leaves a full copy behind until GraphDB runs out of memory. SIS can't be stopped per repository either, because it can't be restarted after shutdown, and GraphDB calls the plugin's
shutdown()on plugin stop/start as well as on deletion.The same cause also limited EPSG support. SIS reads EPSG definitions from an embedded Derby database in
SIS_DATA, and only one copy of Derby per process can open it. Only the first repository to use EPSG data got projected CRS support; the others silently fell back to SIS's built-in subset.Fix
The plugin's code and dependencies (Jena, SIS, Derby, Lucene) are now loaded once per GraphDB process and shared by all repositories. GraphDB still loads a small entry point for each repository, which passes its calls to that repository's plugin instance in the shared runtime. Each repository keeps its own plugin instance, configuration and spatial index.
As a result, deleting a repository releases its copy of the plugin, SIS and Derby run once, and every repository can use the EPSG data.