test: make workspace-switch and kit-registry concurrency tests deterministic - #870
Conversation
PR SummaryLow Risk Overview
The Reviewed by Cursor Bugbot for commit 0a33db8. Bugbot is set up for automated code reviews on this repo. Configure here. |
📦 SDK Size Impact ReportMeasures how much the SDK adds to an app's size (with-SDK minus without-SDK).
➡️ SDK size impact change is minimal. Raw measurementsTarget branch (main): {"baseline_app_size_kb":84,"baseline_executable_size_bytes":75464,"with_sdk_app_size_kb":1936,"with_sdk_executable_size_bytes":76312,"sdk_impact_kb":1852,"sdk_executable_impact_bytes":848,"xcframework_size_kb":6664}This PR: {"baseline_app_size_kb":84,"baseline_executable_size_bytes":75464,"with_sdk_app_size_kb":1936,"with_sdk_executable_size_bytes":76312,"sdk_impact_kb":1852,"sdk_executable_impact_bytes":848,"xcframework_size_kb":6664} |
192ce45 to
5006dff
Compare
5006dff to
b0a010a
Compare
…inistic testSwitchWorkspaceKitsWithStop waited on two nested 10-second dispatch_after blocks, so it always burned at least 20 seconds of wall clock and depended on fixed timing rather than on the SDK actually being ready. Wait on the SDK's own readiness signals instead - MParticle.initialized after startWithOptions:, and the replacement shared instance after switchWorkspaceWithOptions: - by spinning the main run loop until the condition holds. The assertions are unchanged, so the test still checks the same behaviour; it now takes 0.11s instead of 20.2s. testActiveKitsRegistryThreadSafety tracked failure in a plain __block BOOL written from four concurrent blocks, which is itself an unsynchronized data race, and called XCTFail from those background queues. Guard the shared state with an NSLock, record only the first exception, and report it from the dispatch_group_notify block on the main queue. This commit does not change the flakiness of testActiveKitsRegistryThreadSafety itself: that crash is a real race in MPKitContainer (flushSerializedKits enumerates and mutates kitsRegistry without kitsSemaphore, reached via the early return in configureKits: before the lock is taken). Reproduced at 2 crashes in 10 runs of that test alone. Fixing it changes core SDK locking, so it is deliberately left for a separate, explicitly-reviewed change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
b0a010a to
0a33db8
Compare
BrandonStalnaker
left a comment
There was a problem hiding this comment.
This looks good but please make sure all the CI succeeds before merging
Brings this branch's native-tests / build-kits CI up to date with main's #867 (stop simulator cloning that starved the daemons one Swift-scheme test blocks on; bound the SPM resolve with a per-attempt watchdog instead of trusting git's own low-speed abort, which does not reliably fire) and #870 (the workspace-switch tests moved off nested 10-second dispatch_after chains onto MPWaitForCondition, which polls the SDK's own readiness state instead of guessing at a fixed delay). Auto-merged cleanly everywhere except UnitTests/ObjCTests/MParticleTests.m, where both branches had independently rewritten the same five workspace-switch test bodies - main by removing dispatch_after for MPWaitForCondition, this branch by adding migration-specific assertions (resetRegistry, kitContainer_PRIVATE, registeredKits.anyObject) on top of the old dispatch_after version. Git's line-based merge "succeeded" there without a conflict marker, but silently interleaved hunks from both rewrites - 10 dispatch_after calls survived alongside 3 MPWaitForCondition calls, split unevenly across the five methods. Replaced that whole block with a hand-reconciled version carrying both: MPWaitForCondition throughout, this branch's own assertions preserved. The one real conflict marker, on the MParticle() class-extension additions, was two property declarations that both belonged. testActiveKitsRegistryThreadSafety merged without incident - this branch's independent registry work (MPKitContainer.swift's registryLock/registry are both static, unlike main's per-instance kitsSemaphore that #916 fixes) never touched the same lines as main's #870 change to that test, which only reworked how the test itself tracks failures across its four dispatch_group blocks. Verified: no duplicate method names introduced, build-for-testing succeeds, and MParticleTests + MPKitContainerTests run 147/147 clean, including all five workspace-switch tests (0.1-0.4s each, versus 20-30s before) and the thread-safety stress test.
…st fixes (#915) * feat: accept native GoDaddy R1 certificate chains (#879) feat(network): accept native GoDaddy R1 certificate chains Add GoDaddy TLS Root CA - R1 to the default certificate set while retaining all existing trust anchors. Verify the embedded certificate against GoDaddy's published SHA-256 fingerprint to prevent accidental pin changes. #agentic * chore: Release v9.4.1 (#881) chore: (release) 9.4.1 Updates version to 9.4.1 across the mParticle ecosystem. * docs: trim AGENTS.md to the non-derivable core * docs: correct the disabled-workflow and fork-PR notes * docs: state the Xcode-pin split without pinning versions in prose * docs: correct the disabled-workflow gotcha in AGENTS.md release-ecosystem-from-main.yml is registered in Actions as disabled_manually but its file is not on main at all - only on unmerged branches - so describing it as a workflow file in the tree that never runs was wrong. The real trap is that the workflow list and the tree disagree in both directions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: record how to place a red or cancelled check Observed on this branch: the build-kits Pod Lint jobs fail as a batch on a CocoaPods CDN error, and a cancelled job is a timeout-minutes expiry or a superseded push under pull-request.yml's cancel-in-progress concurrency group - not a test result. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: add CODEOWNERS to mirrored kit subtrees (#885) Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * chore: bump actions/setup-java from 5 to 6 (#886) Bumps [actions/setup-java](https://github.com/actions/setup-java) from 5 to 6. - [Release notes](https://github.com/actions/setup-java/releases) - [Commits](actions/setup-java@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-java dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ci: stop simulator cloning and bound network/test hangs (#867) * ci: stop simulator cloning and bound network/test hangs Two unrelated hangs were holding up CI. Both were unbounded, so they consumed a job's full budget instead of failing. native-tests: the iOS x mParticle-Apple-SDK-Swift cell repeatedly hit its 15-minute timeout. A single test, MPDeviceTests.testDictionaryDescription, took 156-459s across recent runs while every other test finished in under 1.6s; job duration tracked that test exactly. The same test takes 0.069s on tvOS in the same run, so the cause is simulator state, not the assertions. MPDevice reads UIDevice.current for name and identifierForVendor, which are XPC calls into simulator daemons, and mParticle-Apple-SDK-Swift.xcscheme is the only scheme in the repo with parallelizable="YES" - so xcodebuild clones and boots extra iOS simulators on a runner that just erased one, starving the daemons that test then blocks on. Disable parallel testing for the run. The suite is ~2s of tests; locally this is 2x faster end to end (33s vs 74s) because it no longer boots 8 simulator clones. Also wait for a real boot via simctl bootstatus - the action's wait_for_boot only waits for state=Booted, not for first-boot work to finish - and add a per-test timeout so a hung test fails in 2 minutes with an explicit diagnostic instead of silently eating the job timeout. The slowest genuine test on CI is ~30s, leaving 4x headroom. build-kits: Build rokt-sdk-plus-ios stalled 30 minutes inside a git fetch of stripe-ios (2.7 GB, reached transitively via rokt-payment-extension-ios), leaving orphaned git and git-remote-http processes. The job has no timeout-minutes, so only an unrelated concurrency cancel stopped it; otherwise it would have held a macOS runner for the 6-hour default. Make git abort a transfer stalled below 1 KB/s for 3 minutes, retry the resolve three times (matching the existing pod-lint-kits pattern), and add job timeouts so no hang here can run unbounded again. Healthy resolves take 2-5 minutes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: bound each SPM resolve attempt with a kill-and-retry watchdog The first version of this fix relied on git's own low-speed abort to break a stalled fetch. It does not fire: on run 33010942238 the stripe-ios fetch sat for 24.5 minutes with http.lowSpeedLimit/lowSpeedTime set, produced no error, and never returned - so the retry loop never got a turn and the step burned its whole 25-minute budget before failing. Bound each attempt explicitly instead. A watchdog kills the resolve and its surviving git children once an attempt passes ATTEMPT_TIMEOUT_SECONDS (480s, against a healthy 2-5 minute resolve), so a stalled attempt is retried rather than consuming the step. Drop to 2 attempts and a 20-minute step timeout, since an attempt can no longer run unbounded. Keep the git low-speed config: it is not sufficient on its own but still aborts genuinely slow transfers where it does apply. Verified against the extracted step body with a stubbed resolver: a healthy resolve exits 0 in 1s with no kills; a permanent hang is killed twice and fails bounded; a hang followed by a healthy retry is killed once and exits 0. Defaulted the timeout in-place so an unset variable cannot make the watchdog fire immediately. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: run the resolve watchdog cleanup in the parent so it actually executes Bugbot caught that the watchdog's cleanup was dead code. The watchdog TERMed the resolve subshell and then slept 5s before escalating, but TERMing the resolve unblocks the parent's wait immediately, and the parent then kills the watchdog mid-sleep - so the kill -KILL and the pkill of leftover xcodebuild and git-remote-http children never ran. The retry could therefore start while the previous stalled fetch was still alive and race it over SwiftPM's cache locks, which is the exact failure the watchdog was added to prevent. The watchdog now only records that it fired (via a marker file) and sends TERM. Escalation and child cleanup run in the parent after wait returns, where they cannot be pre-empted, followed by a short grace period before the retry. Verified by extracting the step body, stubbing the resolver and replacing the pkill calls with probes: on the old code the cleanup probes fired 0 times out of 4 expected; they now fire 4/4 on a permanent hang and 2/2 when one attempt hangs and the retry succeeds. Cleanup still does not run on a non-timeout failure (0 probes), so a fast failure does not trigger a spurious pkill, and the healthy path is unchanged at exit 0 with no kills. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * test: make workspace-switch and kit-registry concurrency tests deterministic (#870) testSwitchWorkspaceKitsWithStop waited on two nested 10-second dispatch_after blocks, so it always burned at least 20 seconds of wall clock and depended on fixed timing rather than on the SDK actually being ready. Wait on the SDK's own readiness signals instead - MParticle.initialized after startWithOptions:, and the replacement shared instance after switchWorkspaceWithOptions: - by spinning the main run loop until the condition holds. The assertions are unchanged, so the test still checks the same behaviour; it now takes 0.11s instead of 20.2s. testActiveKitsRegistryThreadSafety tracked failure in a plain __block BOOL written from four concurrent blocks, which is itself an unsynchronized data race, and called XCTFail from those background queues. Guard the shared state with an NSLock, record only the first exception, and report it from the dispatch_group_notify block on the main queue. This commit does not change the flakiness of testActiveKitsRegistryThreadSafety itself: that crash is a real race in MPKitContainer (flushSerializedKits enumerates and mutates kitsRegistry without kitsSemaphore, reached via the early return in configureKits: before the lock is taken). Reproduced at 2 crashes in 10 runs of that test alone. Fixing it changes core SDK locking, so it is deliberately left for a separate, explicitly-reviewed change. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * chore: bump minimatch from 3.1.2 to 3.1.5 in /RNExample (#640) Bumps [minimatch](https://github.com/isaacs/minimatch) from 3.1.2 to 3.1.5. - [Changelog](https://github.com/isaacs/minimatch/blob/main/changelog.md) - [Commits](isaacs/minimatch@v3.1.2...v3.1.5) --- updated-dependencies: - dependency-name: minimatch dependency-version: 3.1.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump lodash from 4.17.21 to 4.18.1 in /RNExample (#697) Bumps [lodash](https://github.com/lodash/lodash) from 4.17.21 to 4.18.1. - [Release notes](https://github.com/lodash/lodash/releases) - [Commits](lodash/lodash@4.17.21...4.18.1) --- updated-dependencies: - dependency-name: lodash dependency-version: 4.18.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump @babel/plugin-transform-modules-systemjs from 7.25.9 to 7.29.4 in /RNExample (#767) chore: bump @babel/plugin-transform-modules-systemjs in /RNExample Bumps [@babel/plugin-transform-modules-systemjs](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-transform-modules-systemjs) from 7.25.9 to 7.29.4. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.29.4/packages/babel-plugin-transform-modules-systemjs) --- updated-dependencies: - dependency-name: "@babel/plugin-transform-modules-systemjs" dependency-version: 7.29.4 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump shell-quote from 1.8.1 to 1.10.0 in /RNExample (#799) Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.8.1 to 1.10.0. - [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md) - [Commits](ljharb/shell-quote@v1.8.1...v1.10.0) --- updated-dependencies: - dependency-name: shell-quote dependency-version: 1.10.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: stop the partial mock before releasing the object it wraps MPRoktTests.m diverged too far from main's version for the preceding merge to carry this over as part of it - main's tearDown reorders the same two lines, but the files no longer share enough context for a line-based merge to find that hunk. Applying it directly here instead: releasing self.rokt while OCMPartialMock(self.rokt) is still installed is the wrong order regardless of which branch's file structure it's in. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Thomson Thomas <125323226+thomson-t@users.noreply.github.com> Co-authored-by: mParticle Bot User <developers@mparticle.com> Co-authored-by: Matt Bodle <matt.bodle@rokt.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: James Newman <james.newman@rokt.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matt Bodle <22z33p@gmail.com>
Why
These are the two tests that repeat most often in the ObjC crash failures blocking CI:
MParticleTests.testSwitchWorkspaceKitsWithStopMPKitContainerTests.testActiveKitsRegistryThreadSafetyThey turned out to have two completely different problems, and only one of them is a test problem.
testSwitchWorkspaceKitsWithStop— genuinely a bad testIt waited on two nested hard-coded 10-second
dispatch_afterblocks, so it always burned at least 20 seconds of wall clock and depended on fixed timing rather than on the SDK actually being ready. Under CI load, "10 seconds is surely enough" is exactly the assumption that produces flakes.It now waits on the SDK's own readiness signals —
MParticle.initializedafterstartWithOptions:, then the replacement shared instance afterswitchWorkspaceWithOptions:— by spinning the main run loop until the condition holds, so main-queue work the SDK schedules still runs.The assertions are unchanged. Same behaviour checked, no weakening.
testSwitchWorkspaceKitsWithStopVerified: 3/3 runs of the test alone, and 4/4 runs of the whole
MParticleTestsclass (62 tests, 0 failures).MParticle.initializedis exposed to the test via the existing@interface MParticle ()block, alongside the privates already redeclared there. No product change.testActiveKitsRegistryThreadSafety— the test is right, the SDK is wrongThis one is not flaky tooling. It is correctly catching a real data race, reproduced at 2 crashes in 10 runs of that test on its own, nothing else running.
flushSerializedKits(MPKitContainer.m:217) fast-enumerateskitsRegistryand mutates it throughfreeKit:on the main queue without takingkitsSemaphore— the lock every other accessor of that set uses.activeKitsRegistryreads it under the semaphore;configureKits:mutates it under the semaphore. AndconfigureKits:nilreaches the unlocked flush via an early return before the lock is acquired (MPKitContainer.m:2027) — precisely what the test calls in a loop while three threads read the registry.So the crash is mutation-during-enumeration on shared state, and the test is doing its job.
What this PR does and does not do
Fixed here (test-side defects, real but not the cause of the crash):
encounteredErrorwas a plain__block BOOLwritten from four concurrent blocks — an unsynchronized flag is itself a data race. Now guarded by anNSLock, recording only the first exception.XCTFailwas called from those background queues, which is not safe with XCTest. Failures are now reported from thedispatch_group_notifyblock on the main queue.Not fixed here: the race itself. This test will still flake at roughly its current rate until
MPKitContainerlocking is addressed. That is deliberate — a core-concurrency change in a public SDK does not belong in a CI-stabilization PR, and AGENTS.md asks for coordination on anything touching the kit interface.Notes for whoever takes the race
Groundwork already checked:
flushSerializedKitscallers (mParticle.m:611, mParticle.m:722, MPKitContainer.m:2029) are outside locked regions, so takingkitsSemaphorethere would not deadlock.dispatch_syncto the main queue inside any locked region.freeKit:must stay lock-free:configureKits:already calls it while holding the semaphore, anddispatch_semaphoreis not recursive.freeKit:would still readkitsRegistryunlocked, so it is only a partial fix.Scope left alone
testSwitchWorkspaceOptionsand the other tests in that family still use the 10-secondWORKSPACE_SWITCHING_DELAY(three nested delays, ~30s). The macro is retained for them. The same treatment would cut roughly another 60s off the ObjC suite, but they were not among the tests flagged as flaky, so they are out of scope here.No
CHANGELOG.mdentry: test-only, no consumer-facing SDK change.🤖 Generated with Claude Code