Skip to content

Fix maximize metrics. - #12664

Merged
trivialfis merged 2 commits into
dmlc:masterfrom
trivialfis:fix-maximize-metrics
Oct 7, 2026
Merged

trivialfis merged 2 commits into
dmlc:masterfrom
trivialfis:fix-maximize-metrics

Conversation

@trivialfis

@trivialfis trivialfis commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Close #12612
Close #12621

notes

  • ams is not a valid metric, a ratio is required.
  • No test is added. Any test we add now is just manually copying the list to the test module.
  • All language bindings.

Suggestion

We should deprecate these metrics and make all XGBoost metrics error-based metrics. For existing scores, we can use the inverse or minus.

@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2c199974-d2f2-41d0-88c6-41ad4535a7d4
📥 Commits

Reviewing files that changed from the base of the PR and between 258d402 and d8e8b07.

📒 Files selected for processing (1)
  • R-package/R/callbacks.R

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.


Walkthrough

When maximize is unspecified, Python, R, and JVM metric-direction checks now recognize AMS and interval-regression accuracy as metrics to maximize. The R check also excludes names ending in _mape from its _map match.

Assessment against linked issues

Objective Addressed Explanation
Add ams@ and interval-regression-accuracy to Python early-stopping direction inference when maximize is unspecified [#12612] ✅

Out-of-scope changes

Code Change Explanation
Add metric indicators to the R maximization rules (R-package/R/callbacks.R:691-696) The linked objective specifies Python early stopping and does not state a requirement for the R callback.
Add metric indicators to the JVM maximization rules (jvm-packages/xgboost4j/src/main/java/ml/dmlc/xgboost4j/java/XGBoost.java:38) The linked objective specifies Python early stopping and does not state a requirement for the JVM metric rules.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to d8e8b

The changed callbacks recognize AMS and interval-regression accuracy for automatic maximization, with no supported merge-blocking behavior remaining.

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@trivialfis
trivialfis merged commit c914bd0 into dmlc:master Oct 7, 2026
84 checks passed
@trivialfis
trivialfis deleted the fix-maximize-metrics branch October 7, 2026 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[python-package] Early stopping minimizes the higher-is-better metrics interval-regression-accuracy and ams@k when maximize is not set

2 participants