Skip to content

segmented_vector: fix the const iterator for fancy pointers - #297

Closed
bigerl wants to merge 2 commits into
martinus:mainfrom
dice-group:fix/fancy-pointer-const-iterator
Closed

bigerl wants to merge 2 commits into
martinus:mainfrom
dice-group:fix/fancy-pointer-const-iterator

Conversation

@bigerl

@bigerl bigerl commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

segmented_vector's const iterator declares its block array pointer as const_pointer const*, while the block array is a std::vector<pointer>.

With std::allocator those two are T const* const* and T* const*, and a qualification conversion makes the code compile. With a fancy pointer they are unrelated class types. For a boost::interprocess allocator, and for metall, which builds on the same offset_ptr, they are offset_ptr<T const> const* against offset_ptr<T> const*, so begin() const, cbegin(), end() const, cend() and the converting constructor from the mutable iterator all fail to compile. Every const path of a segmented_map over such an allocator is therefore unusable: find() const, erase(const_iterator), iteration over a const map.

The const iterator now points into pointer const*, which is what m_blocks.data() const returns for every allocator. Constness of the element stays with the reference and pointer member types, so the public iterator interface does not change: *it is still value_type const& and it-> is still const_pointer.

custom_container_boost.cpp only used the mutable paths, so it never saw this. It now iterates the const map, looks a key up through the const map and erases through a const_iterator. That test does not compile without the header change.

How this was checked

clang 21.1.8, C++23, x86_64 Linux.

  • The extended custom_container_boost.cpp compiles with the patch and does not compile without it. Stock 5.0.1 gives
    error: no matching constructor for initialization of 'const_iterator' (aka 'iter_t<true>') at lines 1246, 1249, 1256 and 1259, the four const begin()/end() overloads, and
    error: cannot initialize a member subobject of type 'conditional_t<true, [2 * ...]>' with an lvalue of type 'const conditional_t<false, [2 * ...]>' at line 976, the converting constructor.
  • A separate program puts ankerl::unordered_dense::segmented_map<uint64_t, uint64_t, ..., metall::manager::allocator_type<std::pair<uint64_t, uint64_t>>> into a metall datastore, fills it with 200000 entries, erases a third of them through const iterators, then closes the datastore. A second process blocks one page at the old address of the map with MAP_FIXED_NOREPLACE, so metall has to map the segment somewhere else, and reads everything back: same size, same checksum, and inserting after the reopen works. Segment address 0x71340c069f60 in the first run, 0x72bd90a69f60 in the second.
  • The std::allocator case is unaffected, checked with the same program over segmented_map<uint64_t, uint64_t>.

(prepared with claude)

The const iterator declares its block array pointer as `const_pointer const*`, while the
block array is a `std::vector<pointer>`. With `std::allocator` those are `T const* const*`
and `T* const*`, and a qualification conversion makes the code compile. With a fancy
pointer, e.g. the `boost::interprocess::offset_ptr` that metall hands out, they are
`offset_ptr<T const> const*` and `offset_ptr<T> const*`, two unrelated class types, so
`begin() const`, `cbegin()`, `end() const`, `cend()` and the converting constructor from
the mutable iterator all fail to compile.

The const iterator now points into `pointer const*`, which is what `m_blocks.data() const`
returns for every allocator. Constness of the element stays with the `reference` and
`pointer` member types, so the public iterator interface does not change.
The boost interprocess test only used the mutable paths, so the const iterator of
segmented_vector was never instantiated for an allocator whose pointer is a fancy pointer.
It now iterates the const map, looks a key up through it and erases through a const_iterator.
@bigerl

bigerl commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Closing this: it was opened before our own review was finished. The finding and the fix stand, and we will reopen once we have reviewed it internally. Apologies for the noise.

@bigerl bigerl closed this Sep 18, 2026
@martinus

Copy link
Copy Markdown
Owner

Hi, I went through this before you closed it, so here is what I found. Short version: the change is good and I want it in.

I checked it on a 7950X with clang 22 and gcc 16, boost 1.90, C++17 and C++23. Reverting just the ptr_t line on your branch gives exactly the 5 errors you listed, so the test really does pin the fix. The full suite is 840 cases with 0 failures, and the boost case runs instead of being skipped.

I also wanted to know whether that one line is the whole story. So I wrote a small offset_ptr-like allocator without boost and force instantiated every member of detail::table over it: 0 errors with your patch, the same 5 plus one more without it. As far as I can say this is complete for boost::interprocess and metall.

Two notes, neither of them blocking:

  • Unfortunately the new test only compiles where boost is installed. That is most of the Linux legs, but not Windows, macOS, MinGW or the libc++ leg, which drops libboost-dev on purpose. A boost-free fancy pointer allocator in test/app/ would cover all 34 legs and needs no shared memory segment. I have a 130 line one that reproduces your 5 errors on the old header and passes on the new one, on clang and gcc. I can add that separately so your PR stays as it is.
  • iter_t::operator->() does return &m_data[...][...], so it builds a raw T* and relies on an implicit conversion to pointer. boost::interprocess::offset_ptr has that constructor, which is why neither your test nor metall runs into it. A fancy pointer only has to give you std::pointer_traits<pointer>::pointer_to. With a strict fancy pointer that is the only error left, for both iterators. It is older than your PR, I'll file it separately.

One thing worth writing down somewhere: segmented_map is the only path where a fancy allocator can work at all. map fails inside libstdc++ because __normal_iterator::operator-> returns a raw pointer, so nothing in this header can fix that.

Reopen whenever you're done and I'll merge it.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants