Skip to content

Expose output_dict_columns in pylibcudf ParquetReaderOptions - #24705

Open
mroeschke wants to merge 1 commit into
NVIDIA:mainfrom
mroeschke:pylibcudf-parquet-output-dict-columns
Open

mroeschke wants to merge 1 commit into
NVIDIA:mainfrom
mroeschke:pylibcudf-parquet-output-dict-columns

Conversation

@mroeschke

Copy link
Copy Markdown
Contributor

Description

xref #18908

If cudf-polars moves forward with representing polars.Categorical data using libcudf's dictionary type, this API will be handy for parquet scans

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@mroeschke mroeschke self-assigned this Oct 20, 2026
@mroeschke
mroeschke requested a review from a team as a code owner October 20, 2026 00:41
@mroeschke mroeschke added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 20, 2026
@github-actions github-actions Bot added Python Affects Python cuDF API. pylibcudf Issues specific to the pylibcudf package labels Oct 20, 2026
@coderabbitai

coderabbitai Bot commented Oct 20, 2026

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/cudf/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: 32e1c038-9b81-4d13-ad41-15d943502d36
📥 Commits

Reviewing files that changed from the base of the PR and between d625eff and b816abd.

📒 Files selected for processing: 5
  • python/pylibcudf/pylibcudf/io/parquet.pxd
  • python/pylibcudf/pylibcudf/io/parquet.pyi
  • python/pylibcudf/pylibcudf/io/parquet.pyx
  • python/pylibcudf/pylibcudf/libcudf/io/parquet.pxd
  • python/pylibcudf/tests/io/test_parquet.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.

📝 Summary

Summary by CodeRabbit

  • New Features
    • Added Parquet reader options to return flat string columns as dictionary-encoded columns.
    • Added getter, setter, and builder configuration methods for controlling dictionary-column output.
  • Tests
    • Added coverage verifying dictionary-encoded strings, integer column types, option configuration, and decoded values.

Parquet reader options now support enabling, disabling, and querying dictionary-column output through direct methods or the builder. A test checks that enabled output uses DICTIONARY32 for string columns, preserves INT64 for integer columns, and retains nullable string values after decoding.

Changes: Parquet dictionary-column output

Layer / File(s) Summary
Reader options and validation
python/pylibcudf/pylibcudf/libcudf/io/parquet.pxd, python/pylibcudf/pylibcudf/io/parquet.pxd, python/pylibcudf/pylibcudf/io/parquet.pyx, python/pylibcudf/pylibcudf/io/parquet.pyi, python/pylibcudf/tests/io/test_parquet.py
Reader options add methods to enable and query dictionary-column output. The builder adds a fluent configuration method. The test checks output types and decoded nullable string values.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Suggested reviewers: vyasr

Merge Risk: ⚪ Minimal · up to b816a

No merge-blocking risk was found in the new dictionary-column API; normal build and test checks remain appropriate.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 2 files. (3 skipped: 3… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description check Passed The description explains the purpose of exposing output_dict_columns and links the change to cudf-polars categorical data support. It is related to the changeset.
Title check Passed The title clearly and concisely identifies the main change: exposing output_dict_columns in pylibcudf ParquetReaderOptions.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 2 files. (3 skipped: 3 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improvement / enhancement to an existing function non-breaking Non-breaking change pylibcudf Issues specific to the pylibcudf package Python Affects Python cuDF API.

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant