Skip to content

feat: Support schemata for Schema parsing - #34

Merged
Ten0 merged 11 commits into
Ten0:masterfrom
PookieBuns:feat/schemata
Apr 11, 2026
Merged

Ten0 merged 11 commits into
Ten0:masterfrom
PookieBuns:feat/schemata

Conversation

@PookieBuns

@PookieBuns PookieBuns commented Apr 1, 2026 •

Copy link
Copy Markdown
Contributor

Resolves #30

I want to be transparent that a lot of the test cases were generated by AI (if thats ok with you) using GPT 5.4. However, all the implementation code is hand written and the tests were heavily inspected by myself and pruned for redundancy. Please let me know if you have any questions or if you have a strong stance for AI generated code. If you do not want AI generated code in your repo, I can remove the AI generated test cases and rewrite them, but I find them useful for identifying edge / test cases that I would have overlooked otherwise.

Comment thread serde_avro_fast/src/schema/mod.rs Outdated
Comment on lines +25 to +30
let owned: Vec<String> = schemata_str
.into_iter()
.map(|s| s.as_ref().to_owned())
.collect();
let refs: Vec<&str> = owned.iter().map(String::as_str).collect();
let safe_schema = safe::SchemaMut::from_schemata(schema_str, &refs)?;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure exactly how to eliminate monomorphization for leaf crates other than performing another allocation. Help or direction if this can be optimized would be appreciated!

@Ten0 Ten0 Apr 1, 2026 •

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's almost exactly what I had in mind :)
I think ultimately that allocation is not very expensive considering the cost of the parsing itself, so that's just a nicety.

However I think this API should also be available in SchemaMut, and the code above with that alloc should live there.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated!

Comment thread serde_avro_fast/src/schema/safe/mod.rs Outdated
Comment thread serde_avro_fast/src/schema/safe/parsing/mod.rs
Comment on lines +55 to +58
.ok_or_else(|| SchemaError::new("SchemaKey refers to non-existing node"))?
.as_ref()
.ok_or_else(|| {
SchemaError::new("Remapped SchemaKey refers to an unreachable node")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've thought about whether these should return errors or panic here, since errors from here are only possible (I think) if my implementation is incorrect and corrupts nodes. Am open to suggestions, but currently have it as errors to prevent panicking for end users

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I like panic more here for clarity.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to panic on crate bug instead of producing errors

Comment thread serde_avro_fast/src/schema/safe/remove_unreferenced_nodes.rs Outdated
@PookieBuns
PookieBuns force-pushed the feat/schemata branch 2 times, most recently from 99de48b to d1daedf Compare April 1, 2026 20:22
Comment thread serde_avro_fast/src/schema/mod.rs Outdated
Comment thread serde_avro_fast/src/schema/safe/parsing/mod.rs Outdated
let key_remap = build_remap(&reachable_nodes);
remap_nodes(self, &reachable_nodes, &key_remap)?;
remove_unreachable_nodes(self, &reachable_nodes);
self.schema_json = None;

@Ten0 Ten0 Apr 1, 2026 •

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't affect the json representation AFAICT, so I don't see why we need to lose the original json if there was any here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. remove unreferenced nodes should only be applicable for schemata or self built SchemaMut::from_nodes, and those wouldn't have schema_json to begin with anyways

@PookieBuns
PookieBuns force-pushed the feat/schemata branch 3 times, most recently from 02cac02 to 5220bdd Compare April 2, 2026 00:38
}

#[cfg(test)]
mod tests {

@Ten0 Ten0 Apr 2, 2026 •

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a comment why this test can't live in the usual tests folder.

@PookieBuns PookieBuns Apr 2, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated here

))
})?,
),
schema_json: if schemata_str.is_empty() {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

schemata_str here feels hard to read: I'd expect that the type of this variable is &str, and is_empty() checks whether that &str is empty, but that's not what is happening here.
We should look for a better name.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed for clarity!

@PookieBuns
PookieBuns force-pushed the feat/schemata branch 4 times, most recently from c4820bd to 466ec7a Compare April 2, 2026 02:49
@Ten0
Ten0 merged commit 5833efe into Ten0:master Apr 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for Schemata?

2 participants