CASSANDRA-21094: Use POSIX_FADV_SEQUENTIAL for SSTable reads during compaction and streaming - #5133
Open
cheeeee wants to merge 1 commit into
Open
CASSANDRA-21094: Use POSIX_FADV_SEQUENTIAL for SSTable reads during compaction and streaming#5133cheeeee wants to merge 1 commit into
cheeeee wants to merge 1 commit into
Conversation
…eaming (CASSANDRA-21094) Cassandra performs sequential scans during compaction and streaming via openDataReaderForScan(), but did not previously inform the kernel of this access pattern. On Linux, posix_fadvise with POSIX_FADV_SEQUENTIAL doubles the kernel readahead window (128KB -> 256KB) and triggers asynchronous prefetching ahead of the application buffer fill. This patch implements the dual file descriptor pattern: 1. NativeLibrary: add trySetSequential(fd, offset, len, path) invoking POSIX_FADV_SEQUENTIAL via JNA. 2. FileHandle: apply trySetSequential on the open channel FD when forScan is true and diskAccessMode is not direct, avoiding redundant calls and using cached file().path() without allocations. 3. SSTableReader: when forScan is true in openDataReaderInternal(), allocate a dedicated FileHandle with OnReaderClose.CLOSE_FILE, preventing sequential advice from polluting the main FD used by random point lookups and ensuring clean FD closure when scanning ends. 4. Unit tests: add testSetSequential in NativeLibraryTest and comprehensive readahead window verification in SSTableReaderDataReaderTest proving kernel readahead doubling (32 -> 64 pages) on real SSTable reads. Fixes: CASSANDRA-21094
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jira Issue
Motivation
Cassandra performs sequential scans during compaction and streaming via
openDataReaderForScan(), but does not inform the kernel of this access pattern. On Linux,posix_fadvisewithPOSIX_FADV_SEQUENTIALdoubles the kernel readahead window (e.g. 128KB -> 256KB) and triggers asynchronous prefetching ahead of the application buffer fill.Changes (Dual File Descriptor Pattern)
NativeLibrary: AddedtrySetSequentialoverloads invokingwrappedLibrary.callPosixFadvise(fd, offset, len, POSIX_FADV_SEQUENTIAL).FileHandle: WhenforScan == trueanddiskAccessMode != DiskAccessMode.direct, appliestrySetSequentialon the open channel file descriptor using zero-allocation cachedfile().path().SSTableReader: InopenDataReaderInternal(), whenforScan == true, allocates a dedicatedFileHandlewithOnReaderClose.CLOSE_FILE. This ensures:openDataReader()continues to share the primarydfilechannel with zero extra FD overhead.Testing & Kernel eBPF Verification
NativeLibraryTest: 3/3 tests pass (addedtestSetSequential).SSTableReaderDataReaderTest: 10/10 tests pass (added readahead and dedicated handle lifecycle verification).RandomAccessReaderTest(14/14 pass) andCompactionControllerTest(7/7 pass).kfunc:vmlinux:ext4_readahead):openDataReader()):ra_pages = 32(128KB readahead window).openDataReaderForScan()):ra_pages = 64(256KB readahead window, exactly doubled).