Skip to content

fix(realtime): don't take channel messages for heartbeat replies - #1434

Merged
jan-tennert merged 4 commits into
supabase:masterfrom
sproctor:realtime-heartbeat-ref
Oct 28, 2026
Merged

jan-tennert merged 4 commits into
supabase:masterfrom
sproctor:realtime-heartbeat-ref

Conversation

@sproctor

@sproctor sproctor commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Builds on #1433, now merged into master; master is merged in here.

What changed

  1. A message with ref 0 could be taken for a heartbeat reply. heartbeatRef used 0 to mean "no heartbeat pending", but refs start at 0 on every socket. While no heartbeat was pending, any message carrying ref 0 matched and was dropped as a heartbeat reply. The server's phx_close or phx_error for a socket's first join is one: it carries that join's join_ref, 0, as its ref. And a heartbeat that itself got ref 0 (the first message on a socket with no channels) counted as not pending, so a missing reply to it went unnoticed for an extra interval. The sentinel is now -1 (NO_HEARTBEAT).
  2. Only messages on the phoenix topic are taken for heartbeat replies. Heartbeats are sent on that topic, so a channel message can no longer match the pending heartbeat, whatever its ref.
  3. sendHeartbeat() sends the ref it made. It read the ref back from heartbeatRef to send it, and a reset on connect in between could change it.

#1423 (merged into master after this was opened) resets the heartbeat state when a socket connects. It stored 0, which with this change is a real ref: every new socket would start with heartbeat 0 pending and time out on its first tick. The merge makes it store NO_HEARTBEAT. This branch's own reset in disconnect() is dropped, since every socket goes through connect().

Why it builds on #1433

Swallowing the first join's close hid a race: when a channel rejoins (enabling presence on a subscribed channel, or after a channel error), the server's close for the join it replaced can arrive afterwards, and RCloseEvent removes the channel. #1433 drops messages for an earlier join_ref, as Phoenix's client does. Without it, this fix alone would let a rejoin lose the socket's first channel, which master protected only by accident.

Testing

  • RealtimeHeartbeatTest, on the test clock:
    • a channel close with ref 0 reaches its channel;
    • an unanswered first heartbeat (ref 0) times out on the next tick;
    • fix(realtime): reset the heartbeat ref when a new socket connects #1423's lost-socket test: the next socket isn't torn down for the lost one's unanswered heartbeat. It now uses ScriptedWebsocket (with a new lose()) instead of its own socket, and also answers the new socket's heartbeat and checks that the next one goes out on the same socket.
  • With the connect reset storing 0, the two heartbeat tests fail. With a topic other than phoenix matched, the lost-socket test fails. The ref-0 close test fails on master, where both the topic check and the sentinel are missing.
  • fix(realtime): handle channel errors and rejoins like realtime-js #1433's rejoin tests now send the server's real ref (the join_ref) for its errors and closes, which they couldn't while ref 0 was swallowed.
  • ./gradlew :realtime-kt:jvmTest :realtime-kt:detekt pass.

🤖 Generated with Claude Code

heartbeatRef used 0 for "no heartbeat pending", but refs start at 0. A
message carrying ref 0 while no heartbeat was pending, such as the server's
phx_close or phx_error for a socket's first join, matched it and was dropped
as a heartbeat reply. And a heartbeat that got ref 0 counted as not pending,
so a missing reply to it went unnoticed for an extra interval. Use -1
instead.

disconnect() also left a pending heartbeat set, so the first tick on the
next socket reported a heartbeat timeout and reconnected again. Clear it.

Builds on the realtime channel error fix: once a first join's close is no
longer swallowed, a rejoin relies on its check for an earlier join_ref. The
rejoin tests now send the server's real ref (the join_ref), which they
couldn't while ref 0 was swallowed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sproctor
sproctor force-pushed the realtime-heartbeat-ref branch from a055047 to d388d12 Compare October 23, 2026 02:53
@jan-tennert

Copy link
Copy Markdown
Collaborator

@sproctor Might need to resolve the conflict

@sproctor
sproctor marked this pull request as draft October 27, 2026 13:31
@sproctor

Copy link
Copy Markdown
Contributor Author

Thanks. Yeah, this conflicted with #1423. I'll clean it up.

sproctor and others added 2 commits October 27, 2026 09:46
Brings in supabase#1433, which this branch was stacked on, and supabase#1423, which also
resets the heartbeat state, on connect. Conflict and resolution:

- supabase#1423's reset on connect stored 0, which on this branch is a real ref, so
  every new socket started with heartbeat 0 pending and its first tick timed
  out. It now stores NO_HEARTBEAT, and this branch's reset in disconnect() is
  dropped as redundant: every socket goes through connect().
- Both sides added RealtimeHeartbeatTest; the tests are combined, this
  branch's reading sent messages with Turbine as ScriptedWebsocket now does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
onMessage took any message whose ref matched the pending heartbeat for its
reply, whatever its topic, so the NO_HEARTBEAT sentinel was all that kept
channel messages out. Match the "phoenix" topic too. sendHeartbeat also read
the ref back from heartbeatRef to send it, which a reset on connect could
change in between; send the ref it made.

The lost-socket test from supabase#1423 now uses ScriptedWebsocket (with a new
lose()) instead of its own socket, answers the new socket's heartbeat and
checks that the next one goes out on the same socket. The test for a
heartbeat pending at disconnect() is dropped: a lost socket goes through
disconnect() and connect() too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sproctor
sproctor marked this pull request as ready for review October 27, 2026 13:50
@sproctor
sproctor requested a review from safaorhan as a code owner October 27, 2026 13:50
@jan-tennert

Copy link
Copy Markdown
Collaborator

Thanks, I think the tests broke too

The server sent the close right after "Subscribed", so the channel could
go SUBSCRIBED -> UNSUBSCRIBED before subscribe(blockUntilSubscribed = true)
started waiting, and the wait never ended. Script the server with
ScriptedWebsocket and send the close only once SUBSCRIBED is seen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sproctor

Copy link
Copy Markdown
Contributor Author

Thanks. It was a flaky test. It passed here.

@jan-tennert
jan-tennert merged commit 2383173 into supabase:master Oct 28, 2026
13 checks passed
@sproctor
sproctor deleted the realtime-heartbeat-ref branch October 28, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants