[python-package] Maximize interval-regression-accuracy and ams@k in early stopping

[MohammadHijjawi97]

Fixes #12612.

When maximize is not given, EarlyStopping infers the direction from a hard-coded list of metric prefixes. The built-in higher-is-better metrics interval-regression-accuracy (survival:aft) and ams@k were missing from that list. They were minimized, so training stopped after early_stopping_rounds + 1 rounds with best_iteration = 0. This affects xgb.train, xgb.cv and the sklearn estimators, which all go through EarlyStopping.

This adds "ams@" and "interval-regression-accuracy" to the list. The C++ side only accepts ams in the ams@k form, so the ams@ prefix is enough. I checked the other registered metrics, and none of the remaining ones are higher-is-better.

Added test_early_stopping_maximize_inferred in tests/python/test_callback.py, covering both metrics. It fails without the fix (best_iteration stays 0) and passes with it.

Note: the R package has the same inference in R-package/R/callbacks.R. I kept this PR to the Python package and can follow up there if that's useful.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 513003db-8596-49af-a4a2-6e976f67a378

📥 Commits

Reviewing files that changed from the base of the PR and between a97a7e7 and 45ebffd.

📒 Files selected for processing (2)

• python-package/xgboost/callback.py
• tests/python/test_callback.py

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

Walkthrough

When maximize is unspecified, EarlyStopping now treats ams@ and interval-regression-accuracy as metrics to maximize. Parameterized tests check that early stopping selects the maximum metric value for both metrics.

Assessment against linked issues:

Objective	Addressed	Explanation
Infer the maximizing direction for interval-regression-accuracy [#12612]	✅
Infer the maximizing direction for ams@k [#12612]	✅
Infer the maximizing direction for bare ams [#12612]	❌	The change adds the ams@ prefix but does not add bare ams to the maximization list.

Suggested reviewers: raaif-yousuf

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 45ebf

The Python early-stopping change is mergeable after normal checks; no actionable issue remains identified.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 45ebf

The change corrects model-selection behavior for two existing metrics. It does not appear to add a new access path or privilege, but it changes training outcomes when the metric direction is left unspecified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The affected scope is the existing Python early-stopping decision for matching metric names. The observed state changes remain callback history, model best attributes, and the stop signal.

Trust Boundaries and Controls

• observed — The callback checks that the selected dataset and metric exist in evaluation history before passing the score and metric name to the direction rule. Explicit maximize remains an alternative to inferred direction.

---

_{Comment @coderabbitai help to get the list of available commands.}

[trivialfis]

Thanks. I opened a different PR to fix all language bindings: #12664

[MohammadHijjawi97]

Thanks @trivialfis, fixing it across all bindings in #12664 makes sense. Closing this one in favor of it.