Skip to content

Google SUT doesn't return thinking text - #1664

Merged
bkorycki merged 4 commits into
mainfrom
google-sut-returns-non-thinking-text
Sep 16, 2026
Merged

bkorycki merged 4 commits into
mainfrom
google-sut-returns-non-thinking-text

Conversation

@bkorycki

Copy link
Copy Markdown
Contributor

Bug fix. Sometimes google models return "thinking" parts. We should not be using these as the SUT response.

@bkorycki
bkorycki requested a review from a team as a code owner September 15, 2026 19:40
@bkorycki
bkorycki deployed to Scheduled Testing September 15, 2026 19:40 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@rogthefrog rogthefrog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

Comment thread tests/modelgauge_tests/sut_tests/test_google_genai.py
Base automatically changed from security-v1.1-evaluator-id to main September 16, 2026 18:39
@bkorycki
bkorycki deployed to Scheduled Testing September 16, 2026 19:05 — with GitHub Actions Active
@bkorycki
bkorycki deployed to Scheduled Testing September 16, 2026 19:07 — with GitHub Actions Active
@bkorycki
bkorycki merged commit 3a7f9c2 into main Sep 16, 2026
3 checks passed
@bkorycki
bkorycki deleted the google-sut-returns-non-thinking-text branch September 16, 2026 19:08

This branch was successfully deployed

1 active deployment
Scheduled Testing — a2532190 Deployed Sep 16, 2026 by bkorycki via build #2548
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants