Skip to content

Fix leaked identifiers in H5DSattach_scale() - #6640

Open
hyoklee wants to merge 2 commits into
HDFGroup:developfrom
hyoklee:h5ds-attach-scale-id-leak
Open

Fix leaked identifiers in H5DSattach_scale()#6640
hyoklee wants to merge 2 commits into
HDFGroup:developfrom
hyoklee:h5ds-attach-scale-id-leak

Conversation

@hyoklee

@hyoklee hyoklee commented Aug 25, 2026

Copy link
Copy Markdown
Member

Describe your changes

H5DSattach_scale() rewrites the scale's REFERENCE_LIST attribute whenever a
dataset is attached to a scale that already has one, and that rewrite acquired
three things it never released:

tmp_id = H5Ropen_object(&ndsbuf[j].ref, H5P_DEFAULT, H5P_DEFAULT);
if (tmp_id < 0)
    goto out;
if (H5Rcreate_object(tmp_id, ".", H5P_DEFAULT, &ndsbuf_w[j].ref) < 0) {
    H5Dclose(tmp_id);      /* error path closed it */
    goto out;
}
                           /* success path did not */
  1. the reopened dataset identifier above -- one per reference already attached
    to the scale;
  2. the references read from the old attribute into ndsbuf, which were
    free()d without ever being destroyed;
  3. the reference appended at index nelmts - 1 of ndsbuf_w, because that
    buffer was reclaimed with sid, the old dataspace, which is one element
    shorter than the buffer.

Each of these keeps the file open, which is the symptom in the issue: a later
H5Fcreate() with H5F_ACC_TRUNC on the same file fails with "unable to
truncate a file which is already open"
, an error with nothing in it to point
back at a dimension scale. H5DSdetach_scale() closes the identifier and
reclaims both buffers; attach now does the same.

Reachability

The rewrite loop runs only on the new-style reference path, which
H5DSwith_new_ref() selects when the library is built with
H5_DIMENSION_SCALES_WITH_NEW_REF or when the object's terminal VOL
connector is not the native one. Stacking a pass-through connector over the
native connector does not reach it -- H5VLobject_is_native() compares the
terminal connector class, so pass-through-over-native still counts as native.
An earlier revision of this PR said "only a pass-through VOL connector"; the
comments and the H5DSwith_new_ref() documentation are corrected accordingly.

Test

test_attach_scale_id_leak() in hl/test/test_ds.c attaches one scale to four
datasets, then checks that the scale is the only dataset identifier still open
and that the file can be truncated after being closed. It skips itself on the
old-reference path, where the loop opens nothing. Reverting either fix fails
it:

reverted result
H5Dclose(tmp_id) 7 dataset ID(s) open after attaching, expected 1
the H5Treclaim() calls the truncate fails
nothing passes

All 133 tests pass in a -DHDF5_DIMENSION_SCALES_NEW_REF=ON build; HL_* and
the h5dump/h5ls/h5repack tool tests pass in a default build.

CMake

-DHDF5_DIMENSION_SCALES_NEW_REF=ON was a no-op, so the code path above could
not be exercised from a stock build at all. The option was declared in
hl/CMakeLists.txt, a child scope that the top-level H5pubconf.h
configure_file() never sees, so H5_DIMENSION_SCALES_WITH_NEW_REF was never
defined and libhdf5.settings reported nothing for it. It now lives in
CMakeBuildOptions.cmake, which is included before both src/ and hl/.

Issue ticket number (GitHub or JIRA)

Fixes #6639

Checklist before requesting a review

  • My code conforms to the guidelines in CONTRIBUTING.md
  • I made an entry in release_docs/CHANGELOG.md (bug fixes, new features)
  • I added a test (bug fixes, new features)

🤖 Generated with Claude Code

The loop that recreates the references already stored in the scale's
REFERENCE_LIST attribute reopens each one with H5Ropen_object(), but only
closed the resulting identifier when H5Rcreate_object() failed. On the
success path it was dropped, leaking one dataset ID per reference already
attached to the scale -- and each leaked ID kept its file open, so a later
H5Fcreate() with H5F_ACC_TRUNC on the same file failed with "unable to
truncate a file which is already open".

The matching loop in H5DSdetach_scale() has always closed it there; this
brings attach into line.

Only reachable through a pass-through VOL connector: H5DSwith_new_ref()
sets is_new_ref = (config_flag || !native), so a native object without
H5_DIMENSION_SCALES_WITH_NEW_REF takes the old-style reference branch,
which opens nothing.

Fixes HDFGroup#6639

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Review Checklist

This PR touches the following areas. Each needs a sign-off
from its listed owners before merging.

@hyoklee hyoklee added the Component - C Library Core C library issues (usually in the src directory) label Aug 25, 2026
Closing the reopened dataset ID was not enough to make the file
closeable. Rewriting REFERENCE_LIST also acquired two things it never
released:

  - the references read from the old attribute into ndsbuf were freed
    without being destroyed, and
  - ndsbuf_w was reclaimed with SID, the old dataspace, which is one
    element shorter than the buffer -- the reference just appended at
    index nelmts - 1 was left behind.

Either one keeps the file open, which is the symptom reported in HDFGroup#6639:
a later H5Fcreate(H5F_ACC_TRUNC) on the same file fails with "unable to
truncate a file which is already open". H5DSdetach_scale() reclaims
both; attach now does too.

test_attach_scale_id_leak() in hl/test/test_ds.c attaches one scale to
four datasets and checks that only the scale is still open afterwards
and that the file can then be truncated. Reverting either fix fails it:
without the H5Dclose, 7 dataset IDs are open instead of 1; without the
reclaims, the truncate fails.

The test needs the new-reference path, which is reachable only with
H5_DIMENSION_SCALES_WITH_NEW_REF or behind a connector whose terminal
connector is not native -- H5VLobject_is_native() compares the terminal
class, so stacking the pass-through connector over the native one does
not reach it. The test skips itself on the old-reference path. Comments
and documentation that described the path as "non-native VOL connector"
or "only a pass-through VOL connector" are corrected to say that.

-DHDF5_DIMENSION_SCALES_NEW_REF=ON was itself a no-op: the option was
declared in hl/CMakeLists.txt, a child scope the top-level H5pubconf.h
configure_file() never saw, so H5_DIMENSION_SCALES_WITH_NEW_REF was
never defined and libhdf5.settings reported nothing. Moved to
CMakeBuildOptions.cmake, which is included before src/ and hl/.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@hyoklee
hyoklee requested a review from jhendersonHDF as a code owner August 27, 2026 17:12
@github-actions
github-actions Bot requested review from brtnfld and lrknox and removed request for lrknox and mattjala August 27, 2026 17:12
@hyoklee hyoklee changed the title Fix leaked dataset ID in H5DSattach_scale() Fix leaked identifiers in H5DSattach_scale() Aug 27, 2026
@hyoklee hyoklee added this to the Backlog milestone Aug 28, 2026
@hyoklee
hyoklee removed the request for review from brtnfld August 28, 2026 16:08
@github-actions
github-actions Bot requested review from mattjala and removed request for jhendersonHDF August 28, 2026 16:14
Comment thread CMakeBuildOptions.cmake
# build-settings configure_file() calls
option (HDF5_DIMENSION_SCALES_NEW_REF "Use new-style references with dimension scale APIs" OFF)
mark_as_advanced (HDF5_DIMENSION_SCALES_NEW_REF)
if (HDF5_DIMENSION_SCALES_NEW_REF)

@jhendersonHDF jhendersonHDF Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part and below should be moved back to hl/ so that only the option () and mark_as_advanced () are in this file. The changes in #6617 fix the configuration file so that it uses the option value and doesn't need the variables below. In fact, I think setting DIMENSION_SCALES_WITH_NEW_REF can just be removed. I just need to update the usage in H5build_settings.cmake.c.in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component - C Library Core C library issues (usually in the src directory)

Projects

Status: To be triaged

Development

Successfully merging this pull request may close these issues.

H5DSattach_scale() leaks a dataset ID per existing reference, keeping the file open (pass-through VOL only)

2 participants