Skip to content

[spark] Introduce CTAS support for partitioned tables - #9378

Open
ArnavBalyan wants to merge 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/spark-ctas-partition
Open

[spark] Introduce CTAS support for partitioned tables#9378
ArnavBalyan wants to merge 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/spark-ctas-partition

Conversation

@ArnavBalyan

Copy link
Copy Markdown
Member

Purpose

  • CTAS rejects all partitioned format tables even though Paimon writer supports partitioned writes.
  • Allow partitioned CTAS when the Paimon writer is selected and support writing partition paths and metadata.
  • Continue rejecting the Spark engine writer, which does not support partitioned format table writes in current Spark version.

Tests

  • UT

@ArnavBalyan

Copy link
Copy Markdown
Member Author

cc @JingsongLi @Zouxxyy thanks! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant