Repository navigation
Conversation
The XXH3 snapshot/rollback logic was previously repeated inline for every node/list field of every node type. Move it into helper functions in pg_query_fingerprint.c (_fingerprintChildBegin/End/Node/List), making the generated code shorter and easier to read. In passing, make field_name argument to _fingerprintNode/List be const, avoid unnecessary "if (true)" and fix inconsistent formatting.
Each scalar field is now a single _fingerprint<Kind>Field() call, with the zero/NULL check and value formatting living in pg_query_fingerprint.c instead of being repeated in the generated code for every field. This also fixes two latent issues in the previously generated code: String* fields were dereferenced without a NULL check, and number formatting used sprintf into a fixed 50-byte buffer, which "%f" could overflow for very large double values. Neither is reachable from parse trees produced by the parser itself, but both are reachable via pg_query_fingerprint_node() on trees constructed by other means, e.g. from Protobuf.
Like the RangeVar change before it, per-field custom fingerprint implementations now live in pg_query_fingerprint.c instead of being embedded as C snippets in the generator script. The generator emits a call to _fingerprint<Node>_<field> for these fields.
A new patch (patches/16_gen_node_support_hook.patch) adds a generic --hook option to Postgres' node support generator, which loads a Perl file after parsing the node headers and hands it the collected node type information. This intentionally keeps all libpg_query-specific logic out of the patch. The fingerprint generation lives in scripts/node_support/ and its per-node/per-field policy in scripts/node_support/overrides.pl. This replaces the regex-based header parsing in scripts/generate_fingerprint_outfuncs.rb (now removed) with Postgres' own node struct parser, which understands multi-attribute pg_node_attr annotations, supertypes and typedef aliases. Regenerate with "make node_support".
Port scripts/generate_protobuf_and_funcs.rb to the hook-based generator introduced for fingerprinting, so all node support code is derived from Postgres' own parsing of the node headers, with policy in scripts/node_support/overrides.pl. The srcdata JSON produced by scripts/extract_headers.rb is no longer used in-repo, but remains as a published artifact for downstream projects. Fields Postgres marks read_write_ignore now must have an explicit entry in outfuncs_skip_fields, so a new such field surfaces as a decision instead of silently renumbering protobuf fields.
Node struct definitions now come straight from Postgres' own node header parser, and enum values are evaluated properly (including #define constants and bit-or expressions), fixing fields the regex-based parsing silently dropped (e.g. Query.queryId, Var.varnosyn) and several wrong enum values (AggSplit, ScanDirection, PartitionRangeDatumKind). Comments and non-node structs are no longer included to simplify the new Perl-based logic, and the JSON is now emitted with sorted keys.
To support, ensure we don't rely on macOS-style sed invocation in Makefile.
Replace the hand-written PL/pgSQL JSON dumper (pg_query_json_plpgsql.c)
with output functions generated from Postgres' own parsing of plpgsql.h,
and add pg_query_parse_plpgsql_protobuf, which returns the same trees as
a serialized PLpgSQLParseResult message.
This is a breaking change to the JSON returned by pg_query_parse_plpgsql.
The old dumper was a mechanical port of Postgres' plpgsql_dumptree debug
output, and it was lossy: it emitted a curated subset of fields, inlined
a copy of a variable wherever a struct pointed at one but kept every
other variable reference as a bare index, and silently dropped things
like retvarno (so "RETURN variable;" had no target at all). Every
Postgres upgrade meant hand-checking plpgsql.h for new fields.
The output now follows plpgsql.h. Variables are referenced by their
index into PLpgSQL_function.datums throughout (the pointer fields
become target_dno / var_dno, -1 for none), every compile-time field is
emitted, enums are printed as names, only the polymorphic types keep
their type-name wrapper, and the top level is
{"version":N,"functions":[...]}. Like with regular parse trees, the JSON
is identical to the proto3 JSON mapping of the Protobuf output, which the
C++ backend uses verbatim.
PL/pgSQL structs are currently not nodes in upstream Postgres, and making
them likely introduces a lot of conflicts on updates. Instead our patched
gen_node_support.pl gains a --hook-extra-input option records every struct
of an extra header for the hook without affecting the regular output.
Compiled functions are now freed only after the output has been
produced (the upb backend references their strings directly), and also
when a later statement fails to compile, which previously leaked them.
byval can differ between architectures, so better to not cause differences in parse results (that would fail CI).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.