[FFmpeg-devel] [PATCH v3 0/5] Switch mmaldec to receive_frame API

[Ming Shun Ho]

It would definitely have to be an option.

I tried lowering MAX_DELAYED_FRAMES myself. While it might work for live
video
streams, games streaming in your case, RTSP from CCTV cameras in mine, lower
MAX_DELAYED_FRAMES seems to cause a decoding stall in any video with
re-ordered
frames.

Thanks for testing this patchset!


On Sun, Dec 12, 2021 at 9:09 AM Cameron Gutman <aicommander at gmail.com>
wrote:

> On 12/9/2021 7:01 AM, Andreas Rheinhardt wrote:
> > This is an updated and extended version of [1] by Ho Ming Shun
> > together with some additions by me. Notice that I am unable
> > to test these patches (not even compilation), so I hope that
> > others (Ho Ming Shun) do so.
> >
> > [1]:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285992.html
> >
> > Andreas Rheinhardt (2):
> >    avcodec/mmaldec: Avoid creating unnecessary reference, simplify code
> >    avcodec/mmaldec: Deduplicate AVClasses
> >
> > Ho Ming Shun (3):
> >    avcodec/mmaldec: use decoupled dataflow
> >    avcodec/mmaldec: re-use AVPacket for extra_data
> >    avcodec/mmaldec: fix pointer type warning
> >
> >   libavcodec/mmaldec.c | 81 +++++++++++++++++++++++---------------------
> >   1 file changed, 42 insertions(+), 39 deletions(-)
> >
>
> This is fantastic work! I'm seeing significantly improved decode latency
> here. I've been carrying an out-of-tree hack[0] for years to address the
> extremely poor performance of mmaldec due to this async decoding behavior.
>
> The refactoring in this patch set _almost_ matches my dirty hack.
>
> With a realtime 60 FPS stream (so 1 packet per 16 ms) via Moonlight-Qt[1]:
> - Current master: ~250 ms
> - This patch set: ~18 ms
> - My hack: ~14 ms
>
> With this patchset, I usually end up receiving frame N-1 when submitting
> frame N. My hack manages to avoid asynchronous decoding altogether, so
> I'm always receiving frame N after sending frame N for decoding.
>
> The absolute latency numbers are very close, but going over a frame of
> latency is significant. The effective latency difference when it comes to
> drawing the resulting image is 16 ms on a 60 Hz monitor due to V-sync.
>
> I assume my hack is unsuitable for inclusion in FFmpeg, but perhaps we
> can figure out a way to avoid having to patch FFmpeg to get that behavior.
> Maybe an AVOption to set MAX_DELAYED_FRAMES rather than hardcoding it?
>
> In any case, this is a _major_ improvement for my use so feel free to add
> my Tested-by to this patch series:
>
> Tested-by: Cameron Gutman <aicommander at gmail.com>
>
>
>
> Regards,
> Cam
>
>
> [0]:
> https://github.com/cgutman/FFmpeg/commit/193a0320afd5f316da407208eb2fdceea888ff64
> [1]: https://github.com/moonlight-stream/moonlight-qt
>