[FFmpeg-user] How do I make ffplay play without high latency?

[Jesse Gordon]

On 11/26/2012 11:02 AM, Roger Pack wrote:
>> With mplayer I was using the -benchmark option which makes it play as
>> fast as possible, but even that has just under a second of latency.
> what about -nocache? same?
>

I got mplayer to run once with -nocache but after that I couldn't get it 
to play my stream with -nocache enabled.


>> I know that the video camera is getting the data out the network port
>> quickly because when I use the windows web client, it has a plugin
>> that streams/plays the mpeg4 and it has very low latency -- so I know
>> the camera is not the source of the latency.
> Are you sure the web client is reading the "mp4" version?  the mp4
> version may be introducing latency by forcing a client to wait for a
> "first i-frame" before outputting.


I do know that the windows webclient is reading the mp4 version because 
I ran wireshark and captured the HTTP exchange.
The windows client can play either the mpeg4 or the mjpeg -- and the 
difference in quality is clearly visible.


> http://ffmpeg.org/trac/ffmpeg/wiki/StreamingGuide#Latency might also
> be interesting.


And I finally did an objective test where I video taped the latency of 
the whole video system and looked counted the frames.
"Frames" are NTSC frames - 29.97 or whatever per second.

Note that frame readings could be off by one in either direction.

Windows client, Mpeg4: 5 frames of latency

WIndows client, MJPEG: 9 frames of latency

Linux Firefox, MJPEG: 4 frames of latency

Linux, Mplayer, -benchmark mpeg4: 9 frames of latency

Linux, Mpeg4, ffplay, no options: 13 frames of latency

Linux, mpeg4, ffmpeg ... -f sdl -: 12 frames of latency

Linux, mpeg4, ffplay -fflags nobuffer -vf "setpts=(PTS*0.95)": 13 frames 
of latency

Linux, mpeg4, ffplay -vf "setpts=(PTS*0.95)": 12 frames of latency

I really should have also tried playing the mjpeg via ffplay to see what 
the latency was like, but I'm out of time for tonight.
Perhaps I'll try that another day.

I know the obvious solution is to use mjpeg -- but I cannot because that 
uses a massively larger quantity of network bandwidth which I can't do 
in most situations.

I suppose it could also have to do with operating system network buffer 
sizes.  If (for example) the tcp socket has a 4K buffer, it might hold 
13 frames of video at the low bitrate of the mpeg4 stream. Maybe windows 
has smaller buffers. The mjpeg of course is much higher bitrate, so it 
wouldn't take any time for it to fill up the 4k buffer or whatever.

I'll have to see if my camera supports any UDP mode since UDP packets 
wouldn't be queued up and stored by the operating system - at least not 
like the buffer on a TCP stream.

I really wish for the mpeg4 to work because it gives me realtime 
low-latency video at 30 fps (640x480) over a low enough bitrate that 
it's managable for most internet connections and wireless routers.

I know it sounds silly to be worried about a few frames -- but if you'd 
tried driving something with a half-second delay you'd realize why I'm 
trying to address the issue!

I'm probably at a dead end on this topic, but if anyone has any bright 
ideas I'll eagerly check them out!

Thanks very much,

Jesse Gordon

PS
There is one possibility which I have not explored:
It is possible that the network camera detects the browser client 
strings and changes the encoding latency based on that, however I really 
doubt that such is the case. But I do know that ffplay is fetching the 
exact same URL for the same video.asf "file" on the camera as the 
windows active X plugin fetches.