[AURON #2455] Run UNIX_TIMESTAMP natively for fixed-offset session time zones

[weiqingy]

Which issue does this PR close?

Closes #2455

Rationale for this change

table.local-time-zone accepts fixed-offset ids such as GMT-08:00, and Flink's own validation message recommends that form. The native Flink_UnixTimestamp resolved its session zone by exact-match lookup against the IANA time zone database, which carries no GMT±HH:MM entries, so #2448 rejects those ids at plan time and the whole Calc falls back to Flink.

That gate was the right call, since before it the id reached the native call and failed past the point where any fallback remained. But it costs native execution for a large family of zones, and it is reachable with no explicit configuration at all: the default value resolves to ZoneId.systemDefault(), so a TaskManager running with TZ=GMT-08:00 lands there silently.

One correction to the issue body while I am here. I wrote that 141 of 745 accepted ids are in this family. Enumerating them gives 2160 of 2764. The family is every minute, not quarter-hours: GMT+05:07 is accepted and normalizes to itself. My original count assumed quarter-hour granularity and also dropped the negative sub-hour ids. The design does not change, since 2160 is still small enough to enumerate exhaustively, but the number is wrong in the issue and I would rather correct it than leave it.

Two spellings in the issue body also turn out not to need support. Flink's validator rejects UTC+8 and a bare +05:30 as configured values, so only the GMT±HH:MM form is reachable that way.

What changes are included in this PR?

Native. A ZoneSpec that is either an IANA region or a constant UTC offset. The region arm delegates to resolve_offset_secs unchanged, so the ambiguous and gap handling validated for that path is preserved by construction rather than by retesting. The fixed arm returns the constant, which is what Flink computes for these zones at every instant.

The offset parser is hand-rolled rather than delegating to FixedOffset's FromStr, which discards unconsumed input and would read +08:00:30 as +08:00 with no error.

Plan-time gate. An id is admitted when it is not SystemV/* and is either a database region or a fixed offset. SystemV/* still falls back: those ids are genuinely absent from the database the native lookup consults, so the gate narrows rather than disappears. Both call sites route through the one predicate method, so there is no second site to keep in sync.

Bare ±HH:MM is admitted as well. The default configuration path returns ZoneId.systemDefault() without validating it, so a TaskManager on TZ=EST delivers -05:00, a spelling the validator itself would reject. Three are reachable this way.

A note on why the offset is resolved natively rather than mapped on the JVM side. Mapping onto the database's Etc/GMT∓N names would invert the sign, since Etc/GMT+8 is UTC−8, and an error there ships timestamps wrong by twice the offset rather than merely unaccelerated. Those names are also whole-hours-only across UTC−12 to +14, so +05:45 and everything past ±12 has no equivalent at any sign.

arrow::array::timezone::Tz is worth mentioning because it is exactly this abstraction and is already on the classpath. It does not fit here: its grammar accepts +08:00 but rejects GMT-08:00, which is the entire family in question, and its offset type does not expose the base-UTC-offset accessor that the ambiguous and gap branches depend on, so adopting it would have meant rewriting the semantics this change is trying to leave alone.

Are there any user-facing changes?

Yes. A session on a fixed-offset time zone now runs UNIX_TIMESTAMP natively instead of falling back to Flink. Results are unchanged; only the execution path differs. Sessions on SystemV/* continue to fall back.

How was this patch tested?

The gate and the native parser have to admit exactly the same set. Admitting more would not surface as an error: a failure inside the native call is turned into a default value, so the query would return no rows rather than fail. That agreement was therefore checked over every reachable id, and independently three times, each derivation building both sides differently. The strongest of the three compiled the shipped Rust source directly rather than transcribing it, and derived the reachable set by calling Flink's own config accessor over 92,476 candidate strings rather than modelling it. All three agree: 2754 admitted, 2754 resolvable, zero admitted-but-unresolvable and zero resolvable-but-rejected. The 13-id gap is exactly SystemV/*.

Native tests cover the whole family exhaustively rather than by sampling, with expected offsets recomputed from the loop indices so the assertion is not circular. Because that loop cannot falsify the closed form it is built on, a separate fixture pins 13 spot values taken from a SimpleDateFormat oracle, covering GMT, GMT±00:01, GMT±18:00, GMT+05:07, GMT+05:45, an epoch day crossing, 1900, and 1582. The differential sweep behind those values ran 155,520 comparisons across four JDKs with no mismatches.

The negative cases are the ones that discriminate between a strict parser and a lenient one, so they cover near-misses rather than garbage: second precision, an embedded space, repeated colons, a Unicode minus sign, and a single-digit hour. Out-of-range offsets are covered too, and both guards were mutation-tested: deleting either one admits bad offsets while leaving the other's cases rejected.

On the JVM side the family is enumerated against the gate, SystemV/* is asserted to still fall back, and the integration test now asserts on the fallback counter rather than only on the answer, since native and fallback both produce the right answer and an answer-only assertion cannot tell them apart.

RexCallConverterTest 54 green, AuronFlinkCalcITCase 23 green, native suite 25 green, Checkstyle clean.

Was this patch authored or co-authored using generative AI tooling?

• Yes
• No

Generated-by: Claude Code (Claude Opus 5)

[Copilot]

Pull request overview

This PR extends native execution of Flink UNIX_TIMESTAMP to support fixed-offset session time zones (e.g., GMT-08:00 and reachable bare offsets like -05:00) by parsing offsets natively, while keeping the plan-time gate to continue rejecting SystemV/* zones.

Changes:

• Add a native ZoneSpec abstraction to resolve either an IANA region (chrono_tz::Tz) or a constant FixedOffset, including a strict fixed-offset parser.
• Update native UNIX_TIMESTAMP evaluation to use the new zone abstraction for offset resolution.
• Expand JVM-side tests and IT coverage to assert fixed-offset zones convert/run natively, and that SystemV/* still falls back.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
native-engine/datafusion-ext-functions/src/flink_datetime.rs	Introduces ZoneSpec and strict fixed-offset parsing so native UNIX_TIMESTAMP can resolve GMT±HH:MM / ±HH:MM.
auron-flink-extension/auron-flink-planner/src/main/java/org/apache/auron/flink/table/planner/converter/RexCallConverter.java	Broadens the plan-time zone gate to admit fixed offsets while still excluding SystemV/*.
auron-flink-extension/auron-flink-planner/src/test/java/org/apache/auron/flink/table/runtime/AuronFlinkCalcITCase.java	Updates ITCase to assert fixed-offset sessions execute natively (using the fallback counter).
auron-flink-extension/auron-flink-planner/src/test/java/org/apache/auron/flink/table/planner/converter/RexCallConverterTest.java	Adds exhaustive gate tests for fixed-offset zones, keeps SystemV/* rejection coverage, and verifies bare-offset reachability via systemDefault().

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[weiqingy]

Good catch, fixed in b4965965.

You're right that the number comes from the bundled tzdb rather than from the gate. Every SystemV/* id the sweep reaches is still asserted to fall back; only the count assertion is relaxed, to the non-vacuity check it was there to provide. I also corrected the javadoc above it, which claimed the count was asserted against what the runtime carries, which is exactly what the literal did not do.

I kept the exact count on the sibling fixed-offset sweep. Those ids are generated from the loop bounds rather than read from the JDK, so the number is fixed by the test itself and a narrowed loop would otherwise go unnoticed.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[weiqingy]

Hi @Tartarus0zm, could you please help review this PR when you get a chance? Thanks!