RSDK-14783 components/camera: flaky test fix TestWhyMustTimeoutOnReadRTP subscription torn down by remote shutdown ordering

[viambot]

Problem

TestWhyMustTimeoutOnReadRTP intermittently fails on CI at client_test.go:1062:

Expected: nil
Actual:   'context canceled'

The assertion checks that main's RTP subscription survives the grand remote (remote-2) going offline.

Root cause

Both tests shut remote-2 down with:

test.That(t, remoteRobot2.Close(...), test.ShouldBeNil)   // robot torn down first
test.That(t, remoteWebSvc2.Close(...), test.ShouldBeNil)  // ...still serving until here

setupRealRobot starts a web service that is separate from the robot's own, so between those two
calls a live web server answers requests on behalf of an already dismantled robot — it advertises an
empty resource list. (A real viam-server never exposes this window: localRobot.Close calls
webSvc.Stop() before closing resources.)

If remote-1's robot client refresh lands in that window, it sees the camera as removed rather than
the machine as offline, so instead of the intended markRemoteUnreachable path it:

1. closes its camera client to remote-2 (updateResourceClients → unsubscribeAll),
2. removes remote-2:rtpPassthroughCamera from its resource graph, so it stops advertising it,
3. which makes main's next refresh close its camera client → unsubscribeAll → the test's
   sub.Terminated is canceled with context canceled.

The window is ~200 ms against a 10 s client refresh interval, hence the rare, timing-dependent
failure (more likely on loaded CI runners).

Reproduction

Temporarily shrinking configTickerInterval and the robot client refresh/check intervals to 300 ms
makes the failure deterministic: TestWhyMustTimeoutOnReadRTP and TestGrandRemoteRebooting fail
100% of the time on main, with logs showing exactly the chain above
(Removing resource from remote client → unsubscribeAll terminating sub). With this change both
pass 100% of the time under the same accelerated timers, and they keep passing with the real timers
(verified repeatedly, including under -race alongside the other remote/passthrough tests in the
package).

Fix

Close the web service before the robot, matching real viam-server shutdown ordering (and the order
these tests already use in their deferred cleanup). The parent then observes a clean disconnect,
which only marks the remote's resources unreachable and leaves main's subscription intact.

TestGrandRemoteRebooting has the identical shutdown sequence and the same latent flake (it asserts
the same thing plus continued liveness across remote-2's reboot), so it gets the same fix.

Key: RSDK-14783

Co-authored by Claude agent for Jira.

[github-actions]

Review Required

This PR was opened by Claude and requires a minimum of 1 approving reviews before it can be merged.

Review process:

1. the assignee — as the person who triggered this Claude workflow, you should review the code thoroughly and submit your approval.
2. Request additional reviewers for the remaining required approvals.

Current status: 0/1 approvals.

[Cheuk (cheukt)]

Claude (@claude) remove or shorten the comments

[claude]

Claude finished Cheuk (@cheukt)'s task in 1m 10s —— View job

---

Removing/shortening comments

• Read changed file
• Remove or shorten comments
• Commit and push

Condensed both multi-line comments to single one-liners:

• TestWhyMustTimeoutOnReadRTP: 5 lines → // Web service before robot: matches real viam-server shutdown; prevents remote-1 misreading an empty resource list as removal.
• TestGrandRemoteRebooting: 3 lines → // Web service before robot; see TestWhyMustTimeoutOnReadRTP for why order matters.
  | Branch

[Benjamin Rewis (benjirewis)]

Seems fair.