Skip to content

fix(ocf): default missing avro.codec to null - #44

Merged
Ten0 merged 2 commits into
Ten0:masterfrom
pinanek:fix-codec
Oct 6, 2026
Merged

Ten0 merged 2 commits into
Ten0:masterfrom
pinanek:fix-codec

Conversation

@pinanek

@pinanek pinanek commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

According to the Apache Avro specification, the avro.codec metadata entry is optional. When it is absent, readers should treat the codec as null.

The current OCF metadata implementation requires avro.codec to be present:

#[derive(serde_derive::Deserialize, serde_derive::Serialize)]
struct Metadata<S, M> {
	#[serde(rename = "avro.schema")]
	schema: S,

	#[serde(rename = "avro.codec")]
	codec: CompressionCodec,

	#[serde(flatten)]
	user_metadata: M,
}

As a result, valid Avro Object Container Files that omit avro.codec fail to deserialize with a missing-field error.

This PR changes CompressionCodec to default to CompressionCodec::Null and marks the codec field with #[serde(default)].

@Ten0

Ten0 commented Oct 5, 2026

Copy link
Copy Markdown
Owner

I'm not fond of the new public Default impl, but agree this should be fixed in the Deserialize impl.
Can you propose a fix that doesn't add this Default impl? (e.g. does Serde allow specifying the default? Otherwise we could just make the field an option since that struct is private anyway)

@Ten0

Ten0 commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Oh wait that is private anyway. OK then my only question is can't we derive default instead of implementing it manually?

@pinanek

pinanek commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Ah, I missed that. Yeah, deriving Default is cleaner. I’ll update it.

@Ten0
Ten0 merged commit 3fa0134 into Ten0:master Oct 6, 2026
3 of 4 checks passed
@Ten0

Ten0 commented Oct 6, 2026

Copy link
Copy Markdown
Owner

released as 2.1.2

@pinanek
pinanek deleted the fix-codec branch October 7, 2026 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants