[Libav-user] Raspberry pi 4 encoding h264_v4l2m2m (Weylus): no picture and warnings

[Alexander Kamm]

Thank you for your time an effort in developing libav...

I want to use libav and ffmpeg together with weylus
(https://github.com/H-M-H/Weylus) on the raspberry pi 4. If I use
libx264 software encoding everything is working except that the
performance is to low for an adequate fps in encoding the screen on the
raspberry pi.

Together with the developer of weylus I'm trying to get the hardware
encoding with h264_v4l2m2m working. Actually the video stream can't be
decoded on the client side and I have some warnings and error in the log
file on the raspberry pi (see below).

Expected behavior

Screen gets encoded on the rasperry pi with h264_4vl2m2m and decoded in
the browser on the client side.

Actual behavior

Encoding with h262_4vl2m2m delivers a video stream (?) but it can't be
decoded in Firefox or Chromium -> no picture. Other features of weylus
are working as expected (controlling the pi form within the client browser).

What I tried:

To get "repeat parameter sets = 1" working: Patched (see below)
libavcodec / h264_4vl2m2m and build it with the raspbian source package.

different values for gop size and timebase

Actually I have no clue which is the root of the problem. Is it
something that is mentioned in the log or do I miss something else
completely?

Any support is appreciated!

______________________________________________________________________________________________

Log file (partly)

2022-02-23T10:10:27.189023Z  INFO weylus::log: Using device /dev/video11
2022-02-23T10:10:27.189434Z  INFO weylus::log: driver 'bcm2835-codec' on
card 'bcm2835-codec-encode' in mplane mode
2022-02-23T10:10:27.189579Z  INFO weylus::log: requesting formats:
output=YU12 capture=H264
2022-02-23T10:10:27.215520Z TRACE weylus::log: output: YU12 16 buffers
initialized: 1124x0632, sizeimage 01092096, bytesperline 00001152
2022-02-23T10:10:27.221552Z TRACE weylus::log: capture: H264 04 buffers
initialized: 1124x0632, sizeimage 00552960, bytesperline 00000000
2022-02-23T10:10:27.221896Z TRACE weylus::log: Failed to set number of
B-frames: Invalid argument
2022-02-23T10:10:27.222071Z TRACE weylus::log: Failed to get number of
B-frames
2022-02-23T10:10:27.222268Z  WARN weylus::log: Failed to set timeperframe
2022-02-23T10:10:27.222592Z TRACE weylus::log: Encoder: header mode = 0
2022-02-23T10:10:27.222874Z TRACE weylus::log: Encoder: bit rate = 200000
2022-02-23T10:10:27.223056Z TRACE weylus::log: Failed to set frame level
rate control: Invalid argument
2022-02-23T10:10:27.223221Z  WARN weylus::log: Failed to set gop size:
Invalid argument
2022-02-23T10:10:27.223488Z TRACE weylus::log: Encoder: repeat parameter
sets = 1
2022-02-23T10:10:27.223644Z TRACE weylus::log: Encoder Context: id (27),
profile (-99), frame rate(0/1), number b-frames (0), gop size (12), bit
rate (200000), qmin (-1), qmax (-1)
2022-02-23T10:10:27.223929Z TRACE weylus::log: Encoder: minimum video
quantizer scale = 0
2022-02-23T10:10:27.224410Z TRACE weylus::log: Encoder: maximum video
quantizer scale = 51
2022-02-23T10:10:27.225093Z DEBUG weylus::log: Empty MOOV enabled;
disabling automatic bitstream filtering
2022-02-23T10:10:27.225614Z DEBUG weylus::log: Not writing 'clli' atom.
No content light level info.
2022-02-23T10:10:27.225836Z DEBUG weylus::log: Not writing 'mdcv' atom.
Missing mastering metadata.
2022-02-23T10:10:27.229606Z  INFO weylus::log: Video:
1124x632 at h264_v4l2m2m pix_fmt: yuv420p
2022-02-23T10:10:32.344507Z ERROR weylus::log: Application provided
invalid, non monotonically increasing dts to muxer in stream 0: 448560 >= 0
2022-02-23T10:10:37.245047Z ERROR weylus::log: Application provided
invalid, non monotonically increasing dts to muxer in stream 0: 889650 >= 0
2022-02-23T10:10:42.185585Z ERROR weylus::log: Application provided
invalid, non monotonically increasing dts to muxer in stream 0: 1335330 >= 0
2022-02-23T10:10:47.063446Z ERROR weylus::log: Application provided
invalid, non monotonically increasing dts to muxer in stream 0: 1774890 >= 0
2022-02-23T10:10:51.928700Z ERROR weylus::log: Application provided
invalid, non monotonically increasing dts to muxer in stream 0: 2212740 >= 0

...

_________________________________________________________________________--

Hard- / Software

Server system:

Raspberry pi 4, 4Gb

Raspbian 64 bit

ffmpeg 4.3.3

Client system

Surface 5

Linux Mint 20.04

Firefox 97.1 (64 bit)

Chromium 98.0.4758.102 (Offizieller Build) for Linux Mint (64-Bit)

____________________________________________________________________________

Patch:

diff --git a/libavcodec/v4l2_m2m_enc.c b/libavcodec/v4l2_m2m_enc.c
index f644b50133..4b2ca7bdf4 100644
--- a/libavcodec/
+++ b/libavcodec/v4l2_m2m_enc.c
@@ -197,6 +197,7 @@ static int v4l2_prepare_encoder(V4L2m2mContext *s)
      v4l2_set_ext_ctrl(s, MPEG_CID(BITRATE) , avctx->bit_rate, "bit
rate", 1);
      v4l2_set_ext_ctrl(s, MPEG_CID(FRAME_RC_ENABLE), 1, "frame level
rate control", 0);
      v4l2_set_ext_ctrl(s, MPEG_CID(GOP_SIZE), avctx->gop_size,"gop
size", 1);
+    v4l2_set_ext_ctrl(s, MPEG_CID(REPEAT_SEQ_HEADER), 1,"repeat
parameter sets", 1);

      av_log(avctx, AV_LOG_DEBUG,
          "Encoder Context: id (%d), profile (%d), frame rate(%d/%d),
number b-frames (%d), "