Skip to content

Rtt fixes - #3208

Open
bgoRelatel wants to merge 3 commits into
signalwire:masterfrom
bgoRelatel:rtt-fixes
Open

bgoRelatel wants to merge 3 commits into
signalwire:masterfrom
bgoRelatel:rtt-fixes

Conversation

@bgoRelatel

Copy link
Copy Markdown

Description

Fix real-time text (RFC 4103) relay on bridged calls

Summary

Real-time text is negotiated correctly on both legs of a bridged call but is
never relayed between them. Five separate defects in switch_core_media.c
contribute; each one hides the next, so they were found and fixed in sequence
while getting RTT working on a production deployment.

With all five applied, text relays byte-for-byte in both directions, both for
calls that negotiate RTT in the initial INVITE and for RTT switched on mid-call
by re-INVITE.

Background

RTT (RFC 4103) carries one keystroke per RTP packet, with redundancy per
RFC 2198 so a lost packet does not permanently lose characters. On a bridged
call FreeSWITCH terminates two independent text streams and must copy between
them, which is what text_bridge_thread() in switch_ivr_bridge.c does.

1. The text m-line's direction attribute is never parsed

switch_core_media_negotiate_sdp() derives the remote media flow from
m->m_mode for audio and for video. The text branch does not, so the text
engine keeps the SWITCH_MEDIA_FLOW_INACTIVE it is reset to before parsing,
regardless of what the far end offered.

That value is copied onto the other leg's send mode when the answer is
processed, after which switch_core_session_write_text_frame() discards every
frame written toward that leg. The discard is logged at DEBUG3 and returns
SWITCH_STATUS_SUCCESS, so nothing surfaces at normal log levels.

Observed: text typed by the callee arrived at the switch and was dropped
inside it, several hundred discard messages per call, nothing on the wire.

2. The RED redundancy level ignores the negotiated fmtp

red_max is fixed at 5, so FreeSWITCH always transmits four redundant
generations plus the primary. The level is negotiated through the fmtp of the
red payload type, where 96/96/96 asks for two generations plus the primary.

Sending more blocks than were agreed was enough for the receiving handsets to
discard the packets. The text arrived correctly encoded, with the right payload
types, in order, with no loss, and was never displayed.

Captures either side of this change show the block count dropping from five to
three, matching what the handsets themselves send, and the text appearing.

3. The text send mode is not set from the offer

The audio and video branches follow set_rmode() with a switch on the
resulting remote mode that sets the local send mode. Text has no equivalent.

This also matters beyond the send mode: that call is the only one that reaches
switch_core_media_set_smode() with SDP_OFFER, which is a precondition of
CF_STREAM_CHANGED and therefore of propagating a mid-call stream addition to
the other leg at all.

4. A propagated stream change does not carry the text stream

When check_stream_changes() makes the far leg re-offer, that leg has never
parsed an m=text line, so it holds neither CF_WANT_RTT nor CF_RTT. Those
gate the text branch of the SDP generator, so the re-offer is built without
text and the far end is never asked for a text stream.

Observed: the caller's re-INVITE adding m=text was accepted, FreeSWITCH
re-INVITEd the callee, and that re-INVITE contained audio only.

5. A re-INVITE skips text setup entirely

switch_core_media_activate_rtp() jumps from the audio section straight to
video: when a re-INVITE leaves the audio endpoint unchanged, which skips the
text section including creation of the text RTP session and its factory.
Adding RTT mid-call is exactly that case: audio stays put, text appears
alongside it.

The non-re-INVITE path already guards this jump with a goto text when
CF_TEXT_POSSIBLE is set and the text RTP session is not ready. The re-INVITE
path has no such guard, on either of its two exits.

Observed: with the preceding fixes applied, both legs negotiated text
correctly and reported sendrecv in both directions, and every relay write
still failed with text engine not available for processing because
t_engine->tf was NULL. 751 such errors in a single call.

Verification

Found and fixed against 1.10.12 on a production switch carrying ordinary
traffic, with two RTT-capable mobile handsets bridged through it.

Verified by packet capture at the switch, decoding the RED/T.140 payloads in
all four directions:

Both scenarios pass: RTT negotiated in the initial INVITE, and RTT enabled
mid-call by re-INVITE.

These commits are the same changes ported to master, where all five
defects are present. On master they apply cleanly and make core builds, but
RTT has not been exercised at runtime against a master build.

Type of Change

  • [ x] Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Code cleanup / refactor

Related Issues

Testing

  • Added/updated unit tests
  • [x ] Tested manually
  • Tested with live SignalWire credentials (if applicable)

Checklist

  • I have read the CONTRIBUTING guidelines
  • My code follows the project's style guidelines
  • I have added tests for my changes (if applicable)
  • I have updated documentation (if applicable)
  • All existing tests pass

Additional Notes

Mid-call RTT activation additionally requires
rtp_pass_codecs_on_stream_change to be true, since the propagation in
switch_core_media_set_smode() is gated on it. That is existing behaviour and
not changed here, but it was not obvious to me, and without it fixes 3 to 5 have no
effect.

The audio and video branches of switch_core_media_negotiate_sdp() derive
their remote media flow from m->m_mode. The text branch never did, so the
text engine kept the SWITCH_MEDIA_FLOW_INACTIVE it is reset to before
parsing, whatever the far end actually offered.

That value is copied onto the other leg's send mode when the answer is
processed, after which switch_core_session_write_text_frame() discards
every frame written toward that leg.

Seen on a bridged call between two RTT-capable handsets: text from the
callee reached the switch and was dropped inside it, logging "Writing text
to RECVONLY/INACTIVE session" once per frame while nothing was sent on the
wire.
red_max is fixed at 5, so we always transmit four redundant generations
plus the primary. RFC 4103 negotiates the level through the fmtp of the
red payload type, where "96/96/96" asks for two generations plus the
primary.

Sending more blocks than were negotiated was enough for the receiving
handsets to discard the packets: the text arrived correctly encoded, with
the right payload types, in order and without loss, and was never
displayed. Packet captures either side of this change show the block count
dropping from five to three and the text appearing.

Taking the level from the fmtp also stops the ring walking slots that were
never written, whose timestamp offsets are emitted relative to zero.
Switching RTT on during an established call sends a re-INVITE adding an
m=text line. Three things stopped that working.

The text branch did not set the local send mode from the offer, which the
audio and video branches do immediately after setting the remote mode.
That call is also the only one that reaches switch_core_media_set_smode()
with SDP_OFFER, which is a condition of CF_STREAM_CHANGED and therefore of
propagating the new stream to the other leg at all.

The propagated re-offer was then built without the text stream, because
the far leg had never parsed an m=text line and so held neither
CF_WANT_RTT nor CF_RTT, which gate the text branch of the SDP generator.

Finally switch_core_media_activate_rtp() jumps from the audio section
straight to video: on a re-INVITE whose audio endpoint is unchanged,
skipping the text section, so the text RTP session and its factory were
never created. The non-re-INVITE path already guards that jump; the
re-INVITE path did not. Every relay write then failed with "text engine
not available for processing".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant