Skip to content

fix(parser): raise ParserMatchError for malformed format strings (#1191) - #1331

Open
Mukller wants to merge 3 commits into
arrow-py:masterfrom
Mukller:fix/parser-indexerror-malformed-fmt
Open

fix(parser): raise ParserMatchError for malformed format strings (#1191)#1331
Mukller wants to merge 3 commits into
arrow-py:masterfrom
Mukller:fix/parser-indexerror-malformed-fmt

Conversation

@Mukller

@Mukller Mukller commented Jul 29, 2026

Copy link
Copy Markdown

Summary

Fixes #1191.

DateTimeParser.parse() raises a raw IndexError instead of ParserMatchError when given a malformed format string.

Root Cause

Inside parse(), after the regex is successfully matched, the code iterates over fmt_tokens and calls match.group(token) for each one:

for token in fmt_tokens:
    ...
    value = match.group(token)   # raises IndexError for malformed fmt

When the format string is malformed, the regex may match but not define a named capture group for every expected token. match.group(token) then raises IndexError: no such group, which propagates unhandled.

Fix

Wrap the match.group() calls in try/except IndexError and re-raise as ParserMatchError, consistent with how the no-match case is handled just above the loop.

 for token in fmt_tokens:
+    try:
         if token == "Do":
             value = match.group("value")
         elif token == "W":
             value = (match.group("year"), match.group("week"), match.group("day"))
         else:
             value = match.group(token)
+    except IndexError:
+        raise ParserMatchError(
+            f"Failed to match {fmt!r} when parsing {datetime_string!r}."
+        )

Callers that already catch ParserMatchError will now also handle the malformed-format-string case without any changes on their side.

…ed fmt (arrow-py#1191)

DateTimeParser.parse() iterates over fmt_tokens and calls
match.group(token) for each one. When the format string is malformed,
the generated regex may not define a named capture group for every
token. In that case match.group(token) raises IndexError (not KeyError),
which propagates up as an unhandled exception.

Wrap the match.group() block in try/except IndexError and re-raise
as ParserMatchError, consistent with how the no-match case is handled
just above the loop.

@Mukller Mukller left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Bug Trace

DateTimeParser.parse(fmt='[-FFFFFFFFFF...', datetime_string='foo')
  1. _generate_pattern_re(fmt) -> returns (fmt_tokens, pattern)
  2. pattern.search('foo') -> match (malformed fmt produces a regex that
     matches 'foo', but with fewer named groups than tokens)
  3. for token in fmt_tokens:
       match.group(token)   # token has no corresponding named group
       -> IndexError: no such group
  4. IndexError propagates to caller -- not caught anywhere

Fix Correctness

try:
    if token == "Do":
        value = match.group("value")
    elif token == "W":
        value = (match.group("year"), match.group("week"), match.group("day"))
    else:
        value = match.group(token)
except IndexError:
    raise ParserMatchError(
        f"Failed to match {fmt!r} when parsing {datetime_string!r}."
    )
  • IndexError from match.group() means the regex matched the input but didn't define a capture group for the token -- equivalent to a parse failure.
  • ParserMatchError is the documented exception for parse failures, so converting IndexError -> ParserMatchError is the correct fix.
  • The same message format is used as for the no-match case above the loop -- callers see a consistent failure message.

Scope

  • 8 lines changed inside parse(), no other changes.
  • No new imports (ParserMatchError is already in scope).
  • All existing parse() paths are unaffected -- the try/except only fires on the IndexError path, which was previously unhandled.

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (2224255) to head (5aeb4c6).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff            @@
##            master     #1331   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           10        10           
  Lines         2315      2318    +3     
  Branches       358       358           
=========================================
+ Hits          2315      2318    +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

…atchError for Do token)

On locales whose ordinal_day_re does not expose a named value capture group
(e.g. GermanLocale, which inherits the base r"(\d+)"), match.group("value")
inside the Do-token branch previously propagated as a bare IndexError.
After the fix it becomes ParserMatchError.
@Mukller

Mukller commented Jul 29, 2026

Copy link
Copy Markdown
Author

Added a regression test in the latest commit to cover the missing branch.

Root cause of missing coverage: (and any locale that does not override ) inherits the base-class pattern , which has no named capture group. When hits the token it calls — which raises — and our block was never executed by any existing test.

New test ():

This directly exercises the path and should bring patch coverage to 100 %.

@Mukller

Mukller commented Aug 6, 2026

Copy link
Copy Markdown
Author

The only failing CI job () failed due to a transient network error during package installation ( from pip, not a test failure). All other 26 jobs pass. Happy to re-trigger if a maintainer can do so, or this should resolve on the next CI run.

@Mukller

Mukller commented Aug 12, 2026

Copy link
Copy Markdown
Author

Update: I retriggered CI with a new commit to check if the windows-latest (pypy-3.11) failure is transient. It failed again with the same error — IncompleteRead during pip package installation, not during any test execution. This is a GitHub Actions infrastructure/network issue on the Windows+PyPy runner that affects multiple runs. All 26 other CI jobs pass. The fix itself is unaffected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IndexError in DateTimeParser.parse

1 participant