[Libav-user] Reducing h264 decoding latency

[Justas P]

Hello all,

I'm looking for pointers on how to reduce the latency of h.264-based
multimedia stream decoding on Windows platform. I receive a data stream
from a scientific instrument via the UDP multicast, so there is very little
(mainly GOP size) that I can change on the encoding side.

On the client side I am using a bare-bones application based on the code
from "
https://stackoverflow.com/questions/10715170/receiving-rtsp-stream-using-ffmpeg-library".
The only change is a replacement of "avcodec_decode_video2" with
"avcodec_send_packet/avcodec_receive_frame" pair. I'm also using FFmpeg
DLLs compiled on the same machine. My FFmpeg configure options are at the
end of this email [1].

So far I have these settings in my AVFormatContext:
format_ctx->max_delay = 0;
format_ctx->probesize = 64;

Are there any other options that I can tweak to reduce the latency? I'm
fine with having a lower quality picture / higher CPU usage if that reduces
the overall latency.

To test the actual glass-to-glass latency, I'm using an external measuring
instrument. So far my best result is ~182 ms. Using a Windows
PerformanceCounter method [2] I measure some of the critical parts of the
code. For these tests, I'm running the code compiled in x64 Release mode.
The values bellow are sliding window averages over 100 measurements:

1) avcodec_send_packet/avcodec_receive_frame -> ~6-7 ms;
2) sws_scale (YUV -> RGB) -> ~0.8 ms;
3) Rendering (Creating a compatible bitmap, and then rendering in a Hwnd)
-> ~8 ms.

I'm running my benchmarks against a test application (FFmpeg based, no
access to the source code) provided with an instrument, which consistently
delivers a glass-to-glass delay of appx. 138 ms. So even if data
conversion from
the YUV to RGB and rendering in my app was perfect (instantaneous), my
application is still appx. 36 ms. behind.

To answer beforehand why I'm not using the test application provided with
the instrument - I need to do extra processing of the video feed before
rendering it.

Thanks for any input!
~J

[1] - FFmpeg configure options and output
$ ./configure --toolchain=msvc --enable-shared --prefix=/c/ffmpeg
--enable-asm --arch=i686
install prefix            /c/ffmpeg
source path               .
C compiler                cl
C library                 msvcrt
ARCH                      x86 (generic)
big-endian                no
runtime cpu detection     yes
standalone assembly       yes
x86 assembler             nasm
MMX enabled               yes
MMXEXT enabled            yes
3DNow! enabled            yes
3DNow! extended enabled   yes
SSE enabled               yes
SSSE3 enabled             yes
AESNI enabled             yes
AVX enabled               yes
AVX2 enabled              yes
AVX-512 enabled           yes
XOP enabled               yes
FMA3 enabled              yes
FMA4 enabled              yes
i686 features enabled     yes
CMOV is fast              yes
EBX available             no
EBP available             no
debug symbols             yes
strip symbols             no
optimize for size         no
optimizations             yes
static                    no
shared                    yes
postprocessing support    no
network support           yes
threading support         w32threads
safe bitstream reader     yes
texi2html enabled         no
perl enabled              no
pod2man enabled           no
makeinfo enabled          no
makeinfo supports HTML    no

[2] -
https://msdn.microsoft.com/en-us/library/windows/desktop/ms644904(v=vs.85).aspx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20180712/aa24ade3/attachment.html>