firmware upgrade broke ssc378de rtsp stream

[danieltwagner]

I upgraded my ssc378de with imx415 from 2.6.01.09-lite to 2.6.05.16-lite and rtsp streaming broke. I don't see any obvious issues in the log, there's just no stream. I made no configuration changes since, and the preview stream continues to work just fine. Downgrading to 2.6.01.09-lite restored rtsp again.

ffplay -v trace rtsp://<camera>/stream=0 shows

[rtsp @ 0x7f20e8000c80] SDP:
v=0
o=OpenIPC 0 0 IN IP4 0.0.0.0
s=Majestic
c=IN IP4 0.0.0.0
t=0 0
m=video 0 RTP/AVP 96
a=control:video
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=000000;packetization-mode=1;sprop-parameter-sets=,
a=framerate:25

[rtsp @ 0x7f20e8000c80] sdp: v='0'
[rtsp @ 0x7f20e8000c80] sdp: o='OpenIPC 0 0 IN IP4 0.0.0.0'
[rtsp @ 0x7f20e8000c80] sdp: s='Majestic'
[rtsp @ 0x7f20e8000c80] sdp: c='IN IP4 0.0.0.0'
[rtsp @ 0x7f20e8000c80] sdp: t='0 0'
[rtsp @ 0x7f20e8000c80] sdp: m='video 0 RTP/AVP 96'
[rtsp @ 0x7f20e8000c80] sdp: a='control:video'
[rtsp @ 0x7f20e8000c80] sdp: a='rtpmap:96 H264/90000'
[rtsp @ 0x7f20e8000c80] video codec set to: h264
[rtsp @ 0x7f20e8000c80] sdp: a='fmtp:96 profile-level-id=000000;packetization-mode=1;sprop-parameter-sets=,'
[rtsp @ 0x7f20e8000c80] RTP Profile IDC: 0 Profile IOP: 0 Level: 0
[rtsp @ 0x7f20e8000c80] RTP Packetization Mode: 1
[rtsp @ 0x7f20e8000c80] Missing PPS in sprop-parameter-sets, ignoring
[rtsp @ 0x7f20e8000c80] sdp: a='framerate:25'
[rtp @ 0x7f20e8008480] No default whitelist set
[udp @ 0x7f20e8008980] No default whitelist set
[udp @ 0x7f20e8008980] end receive buffer size reported is 786432
[udp @ 0x7f20e8018cc0] No default whitelist set
[udp @ 0x7f20e8018cc0] end receive buffer size reported is 786432
[rtsp @ 0x7f20e8000c80] Sending:
SETUP rtsp://openipc-imx415-front.ox.dtw.sh:554/stream=0/video RTSP/1.0
Transport: RTP/AVP/UDP;unicast;client_port=24060-24061
CSeq: 4
User-Agent: Lavf61.7.100
Authorization: Basic [...]
[rtsp @ 0x7f20e8000c80] line='RTSP/1.0 200 OK'
[rtsp @ 0x7f20e8000c80] line='Server: OpenIPC.org RTSP Server/0.1'
[rtsp @ 0x7f20e8000c80] line='Transport: RTP/AVP/UDP;unicast;client_port=24060-24061'
[rtsp @ 0x7f20e8000c80] line='Session: 2075319537;timeout=60'
[rtsp @ 0x7f20e8000c80] setting jitter buffer size to 500
[rtsp @ 0x7f20e8000c80] hello state=0
[rtsp @ 0x7f20e8000c80] Sending:
PLAY rtsp://openipc-imx415-front.ox.dtw.sh:554/stream=0 RTSP/1.0
Range: npt=0.000-
CSeq: 5
User-Agent: Lavf61.7.100
Session: 2075319537
Authorization: Basic [...]

--
[rtsp @ 0x7f20e8000c80] line='RTSP/1.0 200 OK'
[rtsp @ 0x7f20e8000c80] line='CSeq: 5'
[rtsp @ 0x7f20e8000c80] line='Server: OpenIPC.org RTSP Server/0.1'
[rtsp @ 0x7f20e8000c80] line='Range: npt=now-'

... then just hangs there with no output.

moonfire-nvr said:

retina::codec::h265: Ignoring bad H.265 format-specific-params "profile-level-id=000000;packetization-mode=1": no vps
moonfire_base::clock: opening rtsp://<camera>/stream=0 took 30.00136035s!
moonfire_nvr::streamer: sleeping for 1 s after error err=DEADLINE_EXCEEDED: unable to play stream and get first frame within 30s

[semkye]

Так же обновил 3 из 6 камер на том же сенсоре с тем же процессором, и основной поток пропал.

[flyrouter]

Good afternoon
Thank you for your interest in our project.
Today we conducted several tests and did not encounter any issues on a similar configuration.
Could you please check the work on the latest firmware and confirm or refute the problem?

Thank you.

[flyrouter]

[semkye]

Без обновления прошивки работает нормально:

До обновления прошивки сегодня:

После обновления прошивки сегодня:

Камера без обновлений продолжает работать штатно, обе камеры только что обновлены, обе так же не отдают поток по rrsp, звук при этом передаётся.

При этом второй поток работает нормально.

Извините, время слетело.

[Vixand]

Так же обновил 3 из 6 камер на том же сенсоре с тем же процессором, и основной поток пропал.

As far as we know, the IMX415 sensor should correspond to a processor with the QE suffix at the end, not the DE as stated above. Could this be part of the problem?

[Vixand]

Камера без обновлений продолжает работать штатно, обе камеры только что обновлены, обе так же не отдают поток по rrsp, звук при этом передаётся.

How about checking the video in an alternative player, like MPV, or doing more debugging in the console?

[Vixand]

No problems playing video with ffmpeg and mpv.

ffprobe rtsp://root:123456@172.17.32.105/stream=0

ffprobe version 6.1.1-3ubuntu5 Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.2.0-23ubuntu3)
  configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[rtsp @ 0x618c884bea40] max delay reached. need to consume packet
[rtsp @ 0x618c884bea40] RTP: missed 4 packets
Input #0, rtsp, from 'rtsp://root:123456@172.17.32.105/stream=0':
  Metadata:
    title           : Majestic
  Duration: N/A, start: 0.040011, bitrate: N/A
  Stream #0:0: Video: hevc (Main), yuv420p(tv), 2560x1920, 20 fps, 25 tbr, 90k tbn

[semkye]

Я так же свожу все потоки на запись и распознавание во frigate, где всё точно в соответствии с возникшей после обновления прошивки проблемой и повторяется. Видеопотоки с камер, которые были обновлены, перестали записываться. Таким образом влияние VLC логично можно исключить.

[Vixand]

Okay. Then we'll wait for the processor and sensor to be tested on the test bench. But if you have an Anjvision board, it has a processor with quad memory, and we'll be testing the IMX415 on that.

[semkye]

на всех камерах установлены одинаковые платы, купленные одновременно у одного продавца

Алюминиевый радиатор на процессор уже я поставил дополнительно. На нём изначально намертво приклеен керамический радиатор, снять который без повреждения процессора нельзя.

[danieltwagner]

As far as we know, the IMX415 sensor should correspond to a processor with the QE suffix at the end, not the DE as stated above. Could this be part of the problem?

it was sold to me as an SSC378DE (not QE) and the older DE firmware worked fine; I admit I don't know the differences. If there's anything else I can do on the old firmware (or confirm the chip?) please let me know.

# ipcinfo -c -f -l 
ssc37x
infinity6c
imx415_i2c

# ipctool
The ipctool installed as remote GitHub plugin
---
chip:
  vendor: SigmaStar
  model: SSC37X
  family: infinity6c
  tag: "MVX4##I6C#gd11293fROM########XVMT9\u0003"
board:
  vendor: OpenIPC
  version: 2.6.01.09
ethernet:
  mac: "e8:b7:23:04:98:ca"
rom:
- type: nor
  block: 4K
  partitions:
    - name: boot
      size: 0x40000
      sha1: f404edc6
    - name: env
      size: 0x10000
      sha1: acbd6af9
      contains:
        - name: uboot-env
          offset: 0x0
    - name: kernel
      size: 0x200000
      sha1: 95b8aa16
    - name: rootfs
      size: 0x500000
      sha1: 31dc73f2
    - name: rootfs_data
      size: 0xb0000
      path: /overlay,jffs2,rw
  size: 8M
ram:
  total: 128M
  media: 70M
firmware:
  u-boot: "2015.01 (Aug 20 2024 - 15:02:09)"
  kernel: "5.10.61 (PREEMPT Fri Jan 9 22:47:20 UTC 2026)"
  toolchain: buildroot-gcc-13.3.0
  main-app: majestic
sensors:
- vendor: Sony
  model: IMX415
  control:
    bus: 0
    type: i2c
    addr: 0x34

# cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 0 (v7l)
BogoMIPS	: 8.98
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x1
CPU part	: 0xd04
CPU revision	: 0

Hardware	: SStar Soc (Flattened Device Tree)
Revision	: 0000
Serial		: 0000000000000000

# dmesg | grep -i ssc
OF: fdt: Machine model: INFINITY6C SSC027D-S01A

# cat /proc/device-tree/model
INFINITY6C SSC027D-S01A

[danieltwagner]

OpenIPC/ipctool#21 would suggest that 128MB ram = SSC378DE

[semkye]

Заметил одну особенность. Если в настройках потока Video0 выставить разрешение 1920x1080, то выдеопоток снова работает. То есть получается, что после обновления видеопоток перестал работать на разрешении 4k.

[widgetii]

@semkye's observation is the important one, thank you — it changes the picture completely:

Если в настройках потока Video0 выставить разрешение 1920x1080, то видеопоток снова работает.

The stream depends only on the resolution of Video0, with nothing else changed. That rules out the sensor, the ISP and the RTSP server: the camera is producing and serving a 1080p stream from the same sensor mode it fails to produce a 4K one from. It also matches @danieltwagner's trace, where the session is set up and answered but no video data ever follows.

That points at the video buffer pool rather than at the sensor or the SoC variant (DE vs QE), and it fits the fact that the sub stream, the snapshot and audio all keep working — they are much smaller.

We are reproducing this on an infinity6c board in the lab. To confirm the reading — and this is the part we cannot get from our own hardware, because the sensor there cannot reach 4K — could either of you run the following while Video0 is set back to 4K and the main stream is dead, and paste the output?

cat /proc/cmdline
cat /proc/mi_modules/mi_sys_mma/mma_heap_name0
cat /proc/mi_modules/mi_scl/mi_scl0
cat /proc/mi_modules/mi_venc/mi_venc0
cat /etc/majestic.yaml
dmesg | tail -100

The first two are the ones that settle it: /proc/cmdline says how much memory the video pipeline was given at boot, and mma_heap_name0 says how much of it is left. If avail there is at or near zero at 4K and comfortable at 1080p, that is the whole bug.

One more thing worth a line, if it is not too much trouble: with the log level raised (logLevel: debug under system: in /etc/majestic.yaml, then restart), does the log carry any line mentioning an empty frame from the encoder? That would tell us whether the encoder is starved or never started.

In the meantime 1920x1080 on Video0 is a working setting rather than a diagnostic, so there is no need to leave the camera broken for us.

[widgetii]

Found it, and it is fixed. Thank you both — @semkye's resolution observation is what made it findable, and @danieltwagner's trace is what ruled out the half of the stack it was not in.

What was wrong. On this SoC family the scaler can hand the main stream to the encoder in one of two ways: over a hardware ring, a strip of the picture at a time out of a small dedicated buffer, or as whole frames drawn from the shared video memory pool. A change in April moved the main stream from the first to the second. Nothing about that is visible at 1080p. At 4K a single frame in that pool is roughly 13 MiB, and several are in flight at once — so on a board whose bootargs give the video pipeline 70 MiB, and which is also carrying a sub stream, a snapshot channel and the ISP's own buffers, the allocation simply cannot be satisfied.

When it cannot, nothing reports an error. The encoder channel is created, RTSP accepts the connection, answers DESCRIBE and returns 200 to PLAY — and then no frame ever arrives, because none is ever produced. That is exactly the trace above, and it is why the camera looks healthy from every side while streaming nothing.

The sub stream, the snapshot and audio were never moved, which is why they kept working throughout.

Measured. We could not force 4K on our own bench — the IMX335 there tops out at 2560x1920 — so we loaded every channel at that resolution instead and compared the two configurations on the same board, same settings, five minutes each:

video memory pool	whole frames	hardware ring
peak in use, of 70.00 MiB	46.22 MiB	32.25 MiB
scaler channel peak	27.75 MiB	4.43 MiB
one main-stream frame	7.77 MiB	not held at all
dedicated ring buffer	—	1.95 MiB

Scaling the per-pixel cost from that to 3840x2160 is where the 13 MiB a frame comes from. That part is arithmetic rather than a measurement, which is the one thing still worth confirming on real 4K hardware — see below.

The fix puts the main stream back on the hardware ring for this family. It is on master now and will be in the next nightly build. After updating you should be able to set Video0 back to 4K and leave it there; 1920x1080 is no longer needed as a workaround.

Two asks, both optional:

1. When the nightly lands, could you update one camera, set Video0 back to 4K, and confirm the main stream comes back? That is what closes this.
2. If it is still easy, the cat /proc/mi_modules/mi_sys_mma/mma_heap_name0 output from a 4K camera — either before updating while it is broken, or after updating while it works — would turn the 4K figure above from arithmetic into a measurement. Not needed for the fix, only for the record.

Leaving this open until one of you confirms on hardware.