Fix leaked identifiers in H5DSattach_scale() - #6640
Open
hyoklee wants to merge 2 commits into
Open
Conversation
The loop that recreates the references already stored in the scale's REFERENCE_LIST attribute reopens each one with H5Ropen_object(), but only closed the resulting identifier when H5Rcreate_object() failed. On the success path it was dropped, leaking one dataset ID per reference already attached to the scale -- and each leaked ID kept its file open, so a later H5Fcreate() with H5F_ACC_TRUNC on the same file failed with "unable to truncate a file which is already open". The matching loop in H5DSdetach_scale() has always closed it there; this brings attach into line. Only reachable through a pass-through VOL connector: H5DSwith_new_ref() sets is_new_ref = (config_flag || !native), so a native object without H5_DIMENSION_SCALES_WITH_NEW_REF takes the old-style reference branch, which opens nothing. Fixes HDFGroup#6639 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hyoklee
requested review from
brtnfld,
glennsong09,
lrknox and
mattjala
as code owners
August 25, 2026 20:01
Contributor
Closing the reopened dataset ID was not enough to make the file
closeable. Rewriting REFERENCE_LIST also acquired two things it never
released:
- the references read from the old attribute into ndsbuf were freed
without being destroyed, and
- ndsbuf_w was reclaimed with SID, the old dataspace, which is one
element shorter than the buffer -- the reference just appended at
index nelmts - 1 was left behind.
Either one keeps the file open, which is the symptom reported in HDFGroup#6639:
a later H5Fcreate(H5F_ACC_TRUNC) on the same file fails with "unable to
truncate a file which is already open". H5DSdetach_scale() reclaims
both; attach now does too.
test_attach_scale_id_leak() in hl/test/test_ds.c attaches one scale to
four datasets and checks that only the scale is still open afterwards
and that the file can then be truncated. Reverting either fix fails it:
without the H5Dclose, 7 dataset IDs are open instead of 1; without the
reclaims, the truncate fails.
The test needs the new-reference path, which is reachable only with
H5_DIMENSION_SCALES_WITH_NEW_REF or behind a connector whose terminal
connector is not native -- H5VLobject_is_native() compares the terminal
class, so stacking the pass-through connector over the native one does
not reach it. The test skips itself on the old-reference path. Comments
and documentation that described the path as "non-native VOL connector"
or "only a pass-through VOL connector" are corrected to say that.
-DHDF5_DIMENSION_SCALES_NEW_REF=ON was itself a no-op: the option was
declared in hl/CMakeLists.txt, a child scope the top-level H5pubconf.h
configure_file() never saw, so H5_DIMENSION_SCALES_WITH_NEW_REF was
never defined and libhdf5.settings reported nothing. Moved to
CMakeBuildOptions.cmake, which is included before src/ and hl/.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
github-actions
Bot
requested review from
brtnfld and
lrknox
and removed request for
lrknox and
mattjala
August 27, 2026 17:12
github-actions
Bot
requested review from
mattjala
and removed request for
jhendersonHDF
August 28, 2026 16:14
| # build-settings configure_file() calls | ||
| option (HDF5_DIMENSION_SCALES_NEW_REF "Use new-style references with dimension scale APIs" OFF) | ||
| mark_as_advanced (HDF5_DIMENSION_SCALES_NEW_REF) | ||
| if (HDF5_DIMENSION_SCALES_NEW_REF) |
Collaborator
There was a problem hiding this comment.
This part and below should be moved back to hl/ so that only the option () and mark_as_advanced () are in this file. The changes in #6617 fix the configuration file so that it uses the option value and doesn't need the variables below. In fact, I think setting DIMENSION_SCALES_WITH_NEW_REF can just be removed. I just need to update the usage in H5build_settings.cmake.c.in.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes
H5DSattach_scale()rewrites the scale'sREFERENCE_LISTattribute whenever adataset is attached to a scale that already has one, and that rewrite acquired
three things it never released:
to the scale;
ndsbuf, which werefree()d without ever being destroyed;nelmts - 1ofndsbuf_w, because thatbuffer was reclaimed with
sid, the old dataspace, which is one elementshorter than the buffer.
Each of these keeps the file open, which is the symptom in the issue: a later
H5Fcreate()withH5F_ACC_TRUNCon the same file fails with "unable totruncate a file which is already open", an error with nothing in it to point
back at a dimension scale.
H5DSdetach_scale()closes the identifier andreclaims both buffers; attach now does the same.
Reachability
The rewrite loop runs only on the new-style reference path, which
H5DSwith_new_ref()selects when the library is built withH5_DIMENSION_SCALES_WITH_NEW_REFor when the object's terminal VOLconnector is not the native one. Stacking a pass-through connector over the
native connector does not reach it --
H5VLobject_is_native()compares theterminal connector class, so pass-through-over-native still counts as native.
An earlier revision of this PR said "only a pass-through VOL connector"; the
comments and the
H5DSwith_new_ref()documentation are corrected accordingly.Test
test_attach_scale_id_leak()inhl/test/test_ds.cattaches one scale to fourdatasets, then checks that the scale is the only dataset identifier still open
and that the file can be truncated after being closed. It skips itself on the
old-reference path, where the loop opens nothing. Reverting either fix fails
it:
H5Dclose(tmp_id)7 dataset ID(s) open after attaching, expected 1H5Treclaim()callsAll 133 tests pass in a
-DHDF5_DIMENSION_SCALES_NEW_REF=ONbuild;HL_*andthe
h5dump/h5ls/h5repacktool tests pass in a default build.CMake
-DHDF5_DIMENSION_SCALES_NEW_REF=ONwas a no-op, so the code path above couldnot be exercised from a stock build at all. The option was declared in
hl/CMakeLists.txt, a child scope that the top-levelH5pubconf.hconfigure_file()never sees, soH5_DIMENSION_SCALES_WITH_NEW_REFwas neverdefined and
libhdf5.settingsreported nothing for it. It now lives inCMakeBuildOptions.cmake, which is included before bothsrc/andhl/.Issue ticket number (GitHub or JIRA)
Fixes #6639
Checklist before requesting a review
🤖 Generated with Claude Code