Evaluate adding diff-cover to automatically require added/changed lines to be covered

[lsorber]

Considerations:

1. Is requiring 100% coverage on added/changed lines too strict? Maybe we should go for 50% by default and leave it up to the user to set it higher if they wish.
2. The tool works by comparing against a remote branch like origin/main (the default), which needs to be configured in pyproject.toml. What happens is that branch doesn't exist? Is there a way we can figure this out from Cookiecutter?

https://github.com/Bachmann1234/diff_cover

[sinopeus]

Regarding 1, 50% is a sane default, 100% is unrealistic and optimizes for the wrong targets. A possible solution for point 2 is using a variation on this shell script at the end of the poe test task:

export COMPARE_BRANCH=${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-$(git rev-parse --abbrev-ref HEAD)}
git fetch origin ${COMPARE_BRANCH}:refs/remotes/origin/${COMPARE_BRANCH}
diff-cover --ignore-unstaged --compare-branch $COMPARE_BRANCH -c pyproject.toml reports/coverage.xml

This only works for GitLab CI/CD, but I assume that GitHub Actions exposes a similar environment variable. In the case of an MR or PR, this compares against the target branch, which is what we want. In other cases it compares against the current branch, which in practice means you compute the coverage of the changes staged in Git (similar to how pre-commit ignores unstaged changes by default).

[lsorber]

We could decide to accept the default of main, as it's the default branch name for both GitHub and GitLab now so new projects are likely to be using it. And those that use a different default branch name can always configure it themselves.

Alternatively, maybe would use something like

COMPARE_BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "main")

to avoid depending on CI environment variables.

[sinopeus]

It's a sane default but doesn't cover cases where you can have MRs targeted at other branches than the main branch, e.g. Git Flow, where you typically merge from feature branches into develop or from release branches into main. But a simple, opinionated option like what you propose is great as well 🙂