Rtt fixes - #3208
Open
bgoRelatel wants to merge 3 commits into
Open
Rtt fixes#3208bgoRelatel wants to merge 3 commits into
bgoRelatel wants to merge 3 commits into
Conversation
The audio and video branches of switch_core_media_negotiate_sdp() derive their remote media flow from m->m_mode. The text branch never did, so the text engine kept the SWITCH_MEDIA_FLOW_INACTIVE it is reset to before parsing, whatever the far end actually offered. That value is copied onto the other leg's send mode when the answer is processed, after which switch_core_session_write_text_frame() discards every frame written toward that leg. Seen on a bridged call between two RTT-capable handsets: text from the callee reached the switch and was dropped inside it, logging "Writing text to RECVONLY/INACTIVE session" once per frame while nothing was sent on the wire.
red_max is fixed at 5, so we always transmit four redundant generations plus the primary. RFC 4103 negotiates the level through the fmtp of the red payload type, where "96/96/96" asks for two generations plus the primary. Sending more blocks than were negotiated was enough for the receiving handsets to discard the packets: the text arrived correctly encoded, with the right payload types, in order and without loss, and was never displayed. Packet captures either side of this change show the block count dropping from five to three and the text appearing. Taking the level from the fmtp also stops the ring walking slots that were never written, whose timestamp offsets are emitted relative to zero.
Switching RTT on during an established call sends a re-INVITE adding an m=text line. Three things stopped that working. The text branch did not set the local send mode from the offer, which the audio and video branches do immediately after setting the remote mode. That call is also the only one that reaches switch_core_media_set_smode() with SDP_OFFER, which is a condition of CF_STREAM_CHANGED and therefore of propagating the new stream to the other leg at all. The propagated re-offer was then built without the text stream, because the far leg had never parsed an m=text line and so held neither CF_WANT_RTT nor CF_RTT, which gate the text branch of the SDP generator. Finally switch_core_media_activate_rtp() jumps from the audio section straight to video: on a re-INVITE whose audio endpoint is unchanged, skipping the text section, so the text RTP session and its factory were never created. The non-re-INVITE path already guards that jump; the re-INVITE path did not. Every relay write then failed with "text engine not available for processing".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix real-time text (RFC 4103) relay on bridged calls
Summary
Real-time text is negotiated correctly on both legs of a bridged call but is
never relayed between them. Five separate defects in
switch_core_media.ccontribute; each one hides the next, so they were found and fixed in sequence
while getting RTT working on a production deployment.
With all five applied, text relays byte-for-byte in both directions, both for
calls that negotiate RTT in the initial INVITE and for RTT switched on mid-call
by re-INVITE.
Background
RTT (RFC 4103) carries one keystroke per RTP packet, with redundancy per
RFC 2198 so a lost packet does not permanently lose characters. On a bridged
call FreeSWITCH terminates two independent text streams and must copy between
them, which is what
text_bridge_thread()inswitch_ivr_bridge.cdoes.1. The text m-line's direction attribute is never parsed
switch_core_media_negotiate_sdp()derives the remote media flow fromm->m_modefor audio and for video. The text branch does not, so the textengine keeps the
SWITCH_MEDIA_FLOW_INACTIVEit is reset to before parsing,regardless of what the far end offered.
That value is copied onto the other leg's send mode when the answer is
processed, after which
switch_core_session_write_text_frame()discards everyframe written toward that leg. The discard is logged at
DEBUG3and returnsSWITCH_STATUS_SUCCESS, so nothing surfaces at normal log levels.Observed: text typed by the callee arrived at the switch and was dropped
inside it, several hundred discard messages per call, nothing on the wire.
2. The RED redundancy level ignores the negotiated fmtp
red_maxis fixed at 5, so FreeSWITCH always transmits four redundantgenerations plus the primary. The level is negotiated through the fmtp of the
red payload type, where
96/96/96asks for two generations plus the primary.Sending more blocks than were agreed was enough for the receiving handsets to
discard the packets. The text arrived correctly encoded, with the right payload
types, in order, with no loss, and was never displayed.
Captures either side of this change show the block count dropping from five to
three, matching what the handsets themselves send, and the text appearing.
3. The text send mode is not set from the offer
The audio and video branches follow
set_rmode()with aswitchon theresulting remote mode that sets the local send mode. Text has no equivalent.
This also matters beyond the send mode: that call is the only one that reaches
switch_core_media_set_smode()withSDP_OFFER, which is a precondition ofCF_STREAM_CHANGEDand therefore of propagating a mid-call stream addition tothe other leg at all.
4. A propagated stream change does not carry the text stream
When
check_stream_changes()makes the far leg re-offer, that leg has neverparsed an
m=textline, so it holds neitherCF_WANT_RTTnorCF_RTT. Thosegate the text branch of the SDP generator, so the re-offer is built without
text and the far end is never asked for a text stream.
Observed: the caller's re-INVITE adding
m=textwas accepted, FreeSWITCHre-INVITEd the callee, and that re-INVITE contained audio only.
5. A re-INVITE skips text setup entirely
switch_core_media_activate_rtp()jumps from the audio section straight tovideo:when a re-INVITE leaves the audio endpoint unchanged, which skips thetext section including creation of the text RTP session and its factory.
Adding RTT mid-call is exactly that case: audio stays put, text appears
alongside it.
The non-re-INVITE path already guards this jump with a
goto textwhenCF_TEXT_POSSIBLEis set and the text RTP session is not ready. The re-INVITEpath has no such guard, on either of its two exits.
Observed: with the preceding fixes applied, both legs negotiated text
correctly and reported
sendrecvin both directions, and every relay writestill failed with
text engine not available for processingbecauset_engine->tfwas NULL. 751 such errors in a single call.Verification
Found and fixed against 1.10.12 on a production switch carrying ordinary
traffic, with two RTT-capable mobile handsets bridged through it.
Verified by packet capture at the switch, decoding the RED/T.140 payloads in
all four directions:
Both scenarios pass: RTT negotiated in the initial INVITE, and RTT enabled
mid-call by re-INVITE.
These commits are the same changes ported to master, where all five
defects are present. On master they apply cleanly and
make corebuilds, butRTT has not been exercised at runtime against a master build.
Type of Change
Related Issues
Testing
Checklist
Additional Notes
Mid-call RTT activation additionally requires
rtp_pass_codecs_on_stream_changeto be true, since the propagation inswitch_core_media_set_smode()is gated on it. That is existing behaviour andnot changed here, but it was not obvious to me, and without it fixes 3 to 5 have no
effect.