Skip to content

Remove two sources of nightly CI failure (auto-update modal, urlPrefix mock ordering) - #8138

Merged
andrew-polk merged 2 commits into
masterfrom
fix-nightly-flakiness
Aug 3, 2026
Merged

Remove two sources of nightly CI failure (auto-update modal, urlPrefix mock ordering)#8138
andrew-polk merged 2 commits into
masterfrom
fix-nightly-flakiness

Conversation

@andrew-polk

@andrew-polk andrew-polk commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Two independent fixes for things making the nightly red, found while evaluating the 2026-08-03 nightly. Neither one touches the actual visual-regression hang — that is BL-16612, whose fix is on BL-16612-page-checks-hang (#8110). These just remove the noise around it.

1. Don't show the auto-update dialog in automated runs

On a fresh CI runner Settings.Default.AutoUpdateDialogShown is 0, so WorkspaceView opened the "Auto Update" ReactDialog modally as a startup action. With nobody to dismiss it, it sat on the UI thread in its own nested message loop for the whole run — all three stack captures from the 08-03 nightly show the main thread parked in Form.ShowDialog under ShowAutoUpdateDialogIfNeeded.

Program.RunningE2eTests already exists to suppress exactly this (its own comment: "a dialog nobody can dismiss and hanging the whole run") and covers modal error dialogs and Debug.Assert boxes. This nag dialog was simply never included.

This was not the cause of that night's hang — the visual-regression step passed on other nights with this dialog up — just an independent hazard.

2. Install the talking-book urlPrefix mock before the recorder exists

talkingBookSpec > showTool(checksum=missing, audio=missing, scenario=PreTextBox) => UPDATE failed every nightly since 2026-07-29, comparing http://localhost:3000/... against the expected http://localhost:63315/..., while passing locally both in isolation and under a full-suite run with the same worker count.

63315 is not a real port; it is a literal inside a mock of AudioRecording.urlPrefix that setupForAudioRecordingTests installed on theOneAudioRecorder after awaiting initializeTalkingBookToolAsync. Two things made that fragile:

  • theOneAudioRecorder does not exist until that call creates it (it is an export let populated by getAudioRecorder), so the mock could not have gone on the instance any earlier.
  • The call itself already sets the player's src, so that first src came from the real urlPrefix — a relative URL, which jsdom resolves against its own http://localhost:3000/ base.

setCurrentAudioId only refreshes the player when the audio id changes, so a test whose id was already current never overwrote that stale pre-mock value, and compared :3000 against the mocked :63315. Which ids were already current depended on what earlier work happened to leave behind — hence local pass, CI fail.

Fix: mock the prototype, before initializing, so even the init-time write goes through the mock and no stale value exists to inherit.

One deliberate consequence: the mock now applies to every AudioRecording instance rather than only the singleton. That matches the mock's intent ("in tests, urlPrefix returns this absolute URL") and the suite confirms it.

Verification

  • pnpm typecheck — passed
  • pnpm lint — 0 errors (774 pre-existing repo-wide warnings; none in the changed files)
  • build/agent-dotnet.sh build BloomExe — 0 errors
  • Front-end suite — 578 passed / 5 skipped, unchanged from the pre-fix baseline
  • C# suite — see checks

The full-suite numbers being unchanged is the honest reading: it confirms no regression, not that the CI failure is fixed. That test never reproduced locally, so fix 2 is justified by mechanism, and confirming it needs a nightly (or workflow_dispatch) run on this branch. Fix 1 likewise can only be observed in CI.

No tracker card — this branch carries no BL- id (small CI cleanups). Related context: BL-16612.

Devin review


This change is Reviewable

andrew-polk and others added 2 commits August 3, 2026 10:10
On a fresh CI runner Settings.Default.AutoUpdateDialogShown is 0, so
WorkspaceView opened the "Auto Update" ReactDialog modally as a startup
action. With nobody to dismiss it, it sat on the UI thread in its own
nested message loop for the entire run: the 2026-08-03 nightly's hang
dumps show all three captures, eight minutes apart, with the main thread
parked in Form.ShowDialog under ShowAutoUpdateDialogIfNeeded.

Program.RunningE2eTests already exists to suppress exactly this -- its
own comment describes "a dialog nobody can dismiss and hanging the whole
run" -- and covers modal error dialogs and Debug.Assert boxes. This nag
dialog was simply never included, so honor the same flag here.

This was NOT the cause of that night's visual-regression failure (that
is BL-16612, and the step passed on other nights with this dialog up),
just an independent hazard found while reading the dumps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
talkingBookSpec's "showTool(checksum=missing, audio=missing,
scenario=PreTextBox) => UPDATE" failed every nightly since 2026-07-29,
comparing http://localhost:3000/... against the expected
http://localhost:63315/..., while passing locally both in isolation and
under a full-suite run with the same worker count.

63315 is not a real port, it is a literal inside a mock of
AudioRecording.urlPrefix, which setupForAudioRecordingTests installed on
theOneAudioRecorder AFTER awaiting initializeTalkingBookToolAsync. Two
things made that ordering fragile:

- theOneAudioRecorder does not exist until that call creates it (it is an
  `export let` populated by getAudioRecorder), so the mock could not have
  been installed on the instance any earlier.
- The call itself already sets the player's src, so that first src was
  built by the REAL urlPrefix: a relative URL, which jsdom resolves
  against its own http://localhost:3000/ base.

setCurrentAudioId only refreshes the player when the audio id CHANGES, so
a test whose id was already current never overwrote that stale pre-mock
value and compared :3000 against the mocked :63315. Whether an id was
already current depended on what earlier work happened to leave behind,
which is why the machine mattered.

Mock the PROTOTYPE, before initializing, so even the init-time write goes
through the mock and no stale value exists to inherit.

Verified: full front-end suite still 578 passed / 5 skipped, unchanged
from before. Since the failure never reproduced locally, confirming the
CI-side fix needs a nightly (or workflow_dispatch) run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@andrew-polk

Copy link
Copy Markdown
Contributor Author

(Claude Opus 5) Consulted Devin on 2026-08-03 (during preflight) up to commit 334ad92e73b693ff0c4cb04d5fce2439eb940b0e.

Result: 0 bugs, 1 Investigate flag, 2 Informational items.

  • The Investigate flag — the vi.restoreAllMocks() at audioRecordingSpec.ts:1632 un-installing the prototype spy for the rest of that file — is mirrored as its own thread, assessed, and resolved there: Remove two sources of nightly CI failure (auto-update modal, urlPrefix mock ordering) #8138 (comment). Short version: it cannot reach talkingBookSpec.ts, where the failing test lives (vitest isolates per file), and nothing after that line in audioRecordingSpec.ts asserts an audio URL — so it is a pre-existing latent weakness, not a defect in this change.
  • The 2 Informational items are not mirrored (low signal). Both agree with the PR description: the prototype mock now also covers ad-hoc AudioRecording instances (deliberate — no assertion depends on the real prefix), and the e2e early return skips the code that would record the dialog as shown (intended — suppress, not consume).

@andrew-polk
andrew-polk marked this pull request as ready for review August 3, 2026 18:04
@andrew-polk
andrew-polk merged commit c7c78f1 into master Aug 3, 2026
2 checks passed
@andrew-polk
andrew-polk deleted the fix-nightly-flakiness branch August 3, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants