Skip to content

fix: don't let a degenerate .dvcignore line take down every command - #11106

Open
adarshsm wants to merge 1 commit into
treeverse:mainfrom
adarshsm:fix/dvcignore-degenerate-pattern
Open

adarshsm wants to merge 1 commit into
treeverse:mainfrom
adarshsm:fix/dvcignore-degenerate-pattern

Conversation

@adarshsm

Copy link
Copy Markdown

Fixes #11104

The bug

A .dvcignore containing a single bare ! makes every command in the repo fail:

$ git init repro && cd repro && dvc init -q
$ printf '!\n' > .dvcignore
$ dvc status
ERROR: unexpected error - Invalid git pattern: '!': Pattern normalized to nothing.
$ echo $?
255

Git accepts the same line without a word. dvc add and dvc data status fail identically, and a nested .dvcignore does it too — the reporter hit this on a tree built from psf/black's ignore file, which has a bare ! in it.

Root cause

DvcIgnorePatterns.__init__ already knows a pattern can compile to nothing; that is what the existing guard is for:

regex, ignore = GitIgnoreSpecPattern.pattern_to_regex(pattern_info.patterns)
if regex is not None and ignore is not None:
    ...

pathspec signals "this line is a no-op" two different ways, though. For most such lines it returns (None, None), which that guard handles. For a few it raises GitIgnorePatternError instead, and nothing catches it — so those three spellings of the same condition escape as a crash rather than being dropped.

Reproduced on main (56e59829) with pathspec 1.1.1, comparing each line against what git does with it in a .gitignore (rc=1 from git check-ignore means "not ignored", i.e. the line had no effect):

line pattern_to_regex git dvc status before dvc status after
! raises rc=1, silent rc=255 rc=0
! raises rc=1, silent rc=255 rc=0
\ raises rc=1, silent rc=255 rc=0
`` (empty) (None, None) rc=1, silent rc=0 rc=0
#x (None, None) rc=1, silent rc=0 rc=0
/ (None, None) rc=1, silent rc=0 rc=0
[a- (None, None) rc=1, silent rc=0 rc=0
!!, !/, // compiles rc=1, silent rc=0 rc=0

The fix

Catch the error at the one call site and drop the pattern the way the (None, None) case is already dropped, logging a warning that names the offending line:

$ dvc status
WARNING: ignoring invalid pattern in .dvcignore:1:!
There are no data or pipelines tracked in this project yet.

ValueError is the caught type rather than the concrete class because the exception differs between pathspec versions — GitIgnorePatternError on pathspec≥1 and GitWildMatchPatternError on pathspec<1, both ValueError subclasses — and dvc/ignore.py supports both. That keeps the existing version shim from needing a third branch.

Testing

Added test_ignore_degenerate_pattern next to test_ignore_blank_line, parametrized over the three lines. It asserts both halves: the command does not blow up, and the valid pattern on the following line still takes effect (so a fix that swallowed the whole file would not pass).

I confirmed the baseline first — reverting only dvc/ignore.py and re-running gives:

FAILED tests/func/test_ignore.py::test_ignore_degenerate_pattern[!] - pathspe...
FAILED tests/func/test_ignore.py::test_ignore_degenerate_pattern[! ] - pathsp...
FAILED tests/func/test_ignore.py::test_ignore_degenerate_pattern[\\] - pathsp...
3 failed, 27 deselected

With the fix, tests/unit/test_ignore.py is 57 passed and tests/func/test_ignore.py is 30 passed / 3 failed. Those 3 (test_ignored_output, test_ignored_output_nested, test_run_dvcignored_dep) fail identically on a clean checkout of main — they are pre-existing and unrelated to this change.

🤖 Generated with Claude Code

`DvcIgnorePatterns.__init__` already drops a pattern that compiles to
nothing - that is what the `regex is not None and ignore is not None`
guard is for. But `pattern_to_regex` raises for a few spellings of the
same thing instead of returning `(None, None)`, and nothing catches it,
so a `.dvcignore` containing a bare `!`, `! ` or `\` fails every command
in the repo with `unexpected error` and exit 255:

    printf '!\n' > .dvcignore
    dvc status
    ERROR: unexpected error - Invalid git pattern: '!': Pattern normalized to nothing.

Git accepts all of those lines silently, as it does the ones pathspec
reports with `(None, None)`. Catch the error and drop the pattern the
same way, with a warning naming the offending line.

Fixes treeverse#11104

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-project-automation github-project-automation Bot moved this to Backlog in DVC Sep 20, 2026
@codecov

codecov Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.98%. Comparing base (2431ec6) to head (c4bff95).
⚠️ Report is 213 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #11106      +/-   ##
==========================================
+ Coverage   90.68%   90.98%   +0.30%     
==========================================
  Files         504      505       +1     
  Lines       39795    41149    +1354     
  Branches     3141     3263     +122     
==========================================
+ Hits        36087    37439    +1352     
- Misses       3042     3071      +29     
+ Partials      666      639      -27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@KaizenShogun

Copy link
Copy Markdown

I opened #11104, so read this as an interested party measuring your patch, not as a review — I read the diff, I didn't audit it.

I keep an outside conformance bench for .gitignore semantics: the rule files of 33 large repositories, written onto a real tree as .dvcignore, with git check-ignore --no-index (git 2.55.0) as the oracle. Ran it on 56e59829 and on your c4bff95b, nothing else changed, through two entry points — is_ignored_file/is_ignored_dir, and DvcIgnoreFilter.walk(localfs, root).

corpus / entry queries answered divergences from git
8,953 paths / is_ignored_* 8,909 → 8,909 (12 null → 0) 13 → 13
8,953 paths / walk 2,183 → 2,228 0 → 0
1,010 directories / is_ignored_* 1,007 → 1,007 1 → 1
1,010 directories / walk 990 → 1,007 1 → 1

It fixes no divergence and it breaks none. What it moves is crashes into answers, and the repository it unblocks has a name: psf/black, in every row that moves (+45 and +17 on the walk, 12 nulls gone on is_ignored_*). Those 74 newly answered queries all agree with git — the divergence column doesn't move anywhere — so dropping the line isn't only "not dying", it's the verdict git gives.

Worth knowing why this looks small through one door and total through the other. The offending line is nested (black ships it in tests/data/invalid_nested_gitignore_tests/a/.gitignore), and rule files are parsed lazily in _get_trie_pattern. Through is_ignored_* only the 3 queries whose own ancestors reach that directory raise, and the other 42 are answered normally. Through walk the exception escapes the generator and the whole traversal dies — 45 of 45 queries lost. dvc status and dvc add go through the walk.

Prevalence, since it argues against the size of the problem: exactly one of my 33 repositories ships such a line, and black ships it deliberately, in a fixture directory named invalid_nested_gitignore_tests. It is rare. What's out of proportion is the failure mode — one line anywhere in the tree takes out the entire walk, which is what #11104 ran into.

Two smaller things I checked rather than assumed. All three spellings (!, ! , a lone \) raise GitIgnorePatternError, which subclasses ValueError, so the except catches what it needs to. And the warning is not chatty: twice per command whatever the tree size (once per trie built in __init__), because the dropped line never enters pattern_list and so merge_patterns never recompiles it into the children — measured on a 33-directory tree, still 2.

Adapter and corpus: https://github.com/KaizenShogun/gitignore-conformance (adapters/dvc_adapter.py, --entry api|walk).

@adarshsm
adarshsm marked this pull request as ready for review September 22, 2026 12:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A bare ! line in .dvcignore kills every command with unexpected error (git accepts it silently)

2 participants