Skip to content

Report a non-zero exit code if a test file has no tests#794

Merged
jviotti merged 2 commits into
mainfrom
no-tests-fail
Jul 15, 2026
Merged

Report a non-zero exit code if a test file has no tests#794
jviotti merged 2 commits into
mainfrom
no-tests-fail

Conversation

@jviotti

@jviotti jviotti commented Jul 15, 2026

Copy link
Copy Markdown
Member

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

Review in cubic

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
@augmentcode

augmentcode Bot commented Jul 15, 2026

Copy link
Copy Markdown
🤖 Augment PR Summary

Summary: This PR makes the jsonschema test command fail with a non-zero exit status when a test file contains an empty tests array, preventing “no-op” test suites from silently passing.

Changes:

  • Track whether any suite has total == 0 in both text and CTRF reporters.
  • After printing results, throw Fail{EXIT_OTHER_INPUT_ERROR} when an empty suite is detected (exit code 6).
  • Update documentation to note empty tests arrays now lead to non-zero exit.
  • Replace/rename existing “empty tests” fixtures to expect failure, and add a new test covering both text output and --json CTRF output.
  • Adjust an existing config-path test to include a real test case so it continues to pass under the new semantics.

Technical Notes: Behavior is consistent across --verbose and --json modes; output is still emitted, but the process exits non-zero when any suite has zero tests.

🤖 Was this summary useful? React with 👍 or 👎

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread docs/test.markdown Outdated
Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
@jviotti
jviotti merged commit 79fba53 into main Jul 15, 2026
13 of 15 checks passed
@jviotti
jviotti deleted the no-tests-fail branch July 15, 2026 17:22
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Jul 18, 2026
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [sourcemeta/jsonschema](https://github.com/sourcemeta/jsonschema) | minor | `v16.1.0` → `v16.2.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>sourcemeta/jsonschema (sourcemeta/jsonschema)</summary>

### [`v16.2.0`](https://github.com/sourcemeta/jsonschema/releases/tag/v16.2.0)

[Compare Source](sourcemeta/jsonschema@v16.1.0...v16.2.0)

#### What's Changed

- Fix crashes on boolean schemas in various commands by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;792](sourcemeta/jsonschema#792)
- Report a non-zero exit code if a `test` file has no tests by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;794](sourcemeta/jsonschema#794)
- Support top-level custom linter rules by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;795](sourcemeta/jsonschema#795)

**Full Changelog**: <sourcemeta/jsonschema@v16.1.0...v16.2.0>

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNjMuNiIsInVwZGF0ZWRJblZlciI6IjQzLjI2My42IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiLCJhdXRvbWF0aW9uOmJvdC1hdXRob3JlZCIsImRlcGVuZGVuY3ktdHlwZTo6bWlub3IiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant