fix(parser): raise ParserMatchError for malformed format strings (#1191) - #1331
fix(parser): raise ParserMatchError for malformed format strings (#1191)#1331Mukller wants to merge 3 commits into
Conversation
…ed fmt (arrow-py#1191) DateTimeParser.parse() iterates over fmt_tokens and calls match.group(token) for each one. When the format string is malformed, the generated regex may not define a named capture group for every token. In that case match.group(token) raises IndexError (not KeyError), which propagates up as an unhandled exception. Wrap the match.group() block in try/except IndexError and re-raise as ParserMatchError, consistent with how the no-match case is handled just above the loop.
Mukller
left a comment
There was a problem hiding this comment.
Code Review
Bug Trace
DateTimeParser.parse(fmt='[-FFFFFFFFFF...', datetime_string='foo')
1. _generate_pattern_re(fmt) -> returns (fmt_tokens, pattern)
2. pattern.search('foo') -> match (malformed fmt produces a regex that
matches 'foo', but with fewer named groups than tokens)
3. for token in fmt_tokens:
match.group(token) # token has no corresponding named group
-> IndexError: no such group
4. IndexError propagates to caller -- not caught anywhere
Fix Correctness
try:
if token == "Do":
value = match.group("value")
elif token == "W":
value = (match.group("year"), match.group("week"), match.group("day"))
else:
value = match.group(token)
except IndexError:
raise ParserMatchError(
f"Failed to match {fmt!r} when parsing {datetime_string!r}."
)IndexErrorfrommatch.group()means the regex matched the input but didn't define a capture group for the token -- equivalent to a parse failure.ParserMatchErroris the documented exception for parse failures, so convertingIndexError->ParserMatchErroris the correct fix.- The same message format is used as for the no-match case above the loop -- callers see a consistent failure message.
Scope
- 8 lines changed inside
parse(), no other changes. - No new imports (
ParserMatchErroris already in scope). - All existing
parse()paths are unaffected -- the try/except only fires on the IndexError path, which was previously unhandled.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1331 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 10 10
Lines 2315 2318 +3
Branches 358 358
=========================================
+ Hits 2315 2318 +3 ☔ View full report in Codecov by Harness. |
…atchError for Do token)
On locales whose ordinal_day_re does not expose a named value capture group
(e.g. GermanLocale, which inherits the base r"(\d+)"), match.group("value")
inside the Do-token branch previously propagated as a bare IndexError.
After the fix it becomes ParserMatchError.
|
Added a regression test in the latest commit to cover the missing branch. Root cause of missing coverage: (and any locale that does not override ) inherits the base-class pattern , which has no named capture group. When hits the token it calls — which raises — and our block was never executed by any existing test. New test (): This directly exercises the path and should bring patch coverage to 100 %. |
|
The only failing CI job () failed due to a transient network error during package installation ( from pip, not a test failure). All other 26 jobs pass. Happy to re-trigger if a maintainer can do so, or this should resolve on the next CI run. |
|
Update: I retriggered CI with a new commit to check if the |
Summary
Fixes #1191.
DateTimeParser.parse()raises a rawIndexErrorinstead ofParserMatchErrorwhen given a malformed format string.Root Cause
Inside
parse(), after the regex is successfully matched, the code iterates overfmt_tokensand callsmatch.group(token)for each one:When the format string is malformed, the regex may match but not define a named capture group for every expected token.
match.group(token)then raisesIndexError: no such group, which propagates unhandled.Fix
Wrap the
match.group()calls intry/except IndexErrorand re-raise asParserMatchError, consistent with how the no-match case is handled just above the loop.Callers that already catch
ParserMatchErrorwill now also handle the malformed-format-string case without any changes on their side.