[FFmpeg-user] Why does ffplay lie about vq value? (was: How to get ffplay live stream latency under 15 seconds)

[Adam Nielsen]

Hi all,

I've still been working on this but haven't yet worked out how to get
the ffmpeg->network->ffplay latency down to under 15 seconds.

I have however noticed that when playing the video, ffplay includes a
"vq" parameter in the status line, which hovers between 0 KB and 50 KB.
However if I terminate the stream at the source (by exiting ffmpeg),
the "vq" value on the client instantly jumps up to many megabytes, and
gradually counts down to zero as the backlog of ~15 seconds worth of
frames are played (ffplay continues playing the backlog of frames even
though ffmpeg has stopped sending anything).

What would cause the ffplay "vq" value to remain incorrectly near zero
while the stream is active, but then suddenly increase to show its true
value only once new frames have stopped being sent?

All I can think of is that it's something to do with timestamps,
however I have tried to tell ffplay to ignore timestamps and play back
the frames as quickly as possible with "-sync ext" but this doesn't
seem to work.  (See my original message below with more detail.)

Does this vq behaviour give anyone any hints about why ffplay falls so
far behind the live stream?

This is one of the many commands I have used on the Raspberry Pi for
sending the stream, all with the same result:

  ffmpeg -f rawvideo 
    -pix_fmt yuv420p 
    -video_size 1280x960 
    -use_wallclock_as_timestamps 1 
    -fflags +nobuffer 
    -i /dev/video0 
    -vsync cfr
    -r 15
    -c:v h264_v4l2m2m
    -b:v 5M
    -f rtp_mpegts
    udp://239.0.0.1:5004

And on the client x86 PC:

  ffplay -probesize 32 -sync ext udp://239.0.0.1:5004
  # Plus variations of all the options described further below

Many thanks,
Adam.

On Mon, 3 Jan 2022 22:18:55 +1000
Adam Nielsen via ffmpeg-user <ffmpeg-user at ffmpeg.org> wrote:

> Hi all,
> 
> I'm using ffplay to play a video stream from a Raspberry Pi (encoded
> via ffmpeg using the Pi's hardware H264 encoder).
> 
> Unfortunately I cannot work out how to get the stream latency lower
> than about 15 seconds.  I would be happy if the overall delay could be
> reduced to 2 seconds or less.
> 
> I have tried a number of things I have found online but none of them
> seem to make any difference.  I am seeing the following behaviour:
> 
>  * When I start the stream, the video appears within one second (the
>    source stream is configured to send key frames once per second),
>    however the stream is already 5-6 seconds behind the live image.
> 
>  * If I stop the stream at the source, ffplay continues to play the
>    stream (with no new data coming in over the network) up until the
>    timestamp where I stopped it.  So this tells me ffplay must be
>    buffering the data, if it can keep playing for 15 seconds after the
>    stream has stopped transmitting data over the network.
> 
>  * I have told ffplay to play the incoming stream as fast as possible
>    (see below) however it still remains between 5 and 15 seconds
>    behind.  It starts off 5 seconds behind and gradually increases to
>    around 15 seconds behind, but never gets worse than that.  But
>    again, when I stop the stream at the source, ffplay suddenly plays
>    the video at a much higher framerate and catches all the way up to
>    where the live stream was when it stopped being sent over the
>    network.
> 
> I have tried various parameters to ffplay:
> 
> -sync ext
>   Supposed to make the video play as quickly as possible.  Without it
>   the playback can lag further and further behind (beyond 15 seconds)
>   however with it it still lags by up to 15 seconds.
> 
> -probesize 32
>   Makes the video start a little quicker but makes no difference to the
>   latency.
> 
> -vf setpts='N/(30*TB)'
>   Supposed to bump the framerate from the source 15 fps to 30 fps.  This
>   doesn't seem to do anything until the source stream is terminated.
>   At that point it does double the frame rate and quickly renders all
>   the frames ffplay has apparently been buffering the whole time.
> 
> -vf setpts='PTS-STARTPTS'
>   Supposed to remove any offset from the PTS value, but this has no
>   effect.
> 
> -avioflags direct
>   Prevents the video from playing with one message saying "[mpegts]
>   Unable to seek back to the start" followed by many messages stating
>   "[udp] Part of datagram lost due to insufficient buffer size".
> 
> -flags low_delay
> -fflags nobuffer
> -fflags discardcorrupt
> -framedrop
> -strict experimental
> -sync audio
> -sync video
>   None of these make any difference one way or another.
> 
> Is there any way to get ffplay to render everything in its buffer so
> you can consistently get under five seconds latency?  Or does it always
> have to buffer a lot of data and run a long way behind the incoming
> data?
> 
> Cheers,
> Adam.
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> 
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".