[FFmpeg-user] Ultra-low latency streaming

[Federico Salvioli]

Dear all,

     First of all, I would like to thank you all the developers as even if I
am no expert in FFmpeg I have been using it for numerous applications and
your tool is really awesome! Thanks guys.

 

Some time ago I have been asked to setup a streaming server with the lowest
amount of latency possible from a Firewire camera to another computer. The
video feedback is used for teleoperation so the stream has soft real-time
requirements. I have crawled the internet for a while and I saw various
posts on this topic but sometimes maybe too old to be useful or sometimes a
bit generic. I collected and reviewed the following:

 

http://fomori.org/blog/?p=1213-This guy has an interesting article, I will
have a trial to reproduce some of his results.

https://trac.ffmpeg.org/wiki/StreamingGuide

http://ffmpeg.gusari.org/viewtopic.php?f=11
<http://ffmpeg.gusari.org/viewtopic.php?f=11&t=2002> &t=2002

and some others.

 

I ended up setting up a system which looks like the following.

 

  ------------  firewire -----------------------  pipe  -----------  .ffm
------------   rtsp    --------------------

| Camera  |   ==>    |  frame grabber |   => | ffmpeg | => | ffserver |   =>
| ffmpeg/ffplay |

  ------------                 -----------------------
------------         ------------               --------------------

 

I was unable to use the libdc1394 format to capture the feed from the camera
so I used a previously written program to grab the frames and then pipe the
raw yuv420p frames to ffmpeg.

With this setup I managed to obtain around 3 seconds of latency. It is
acceptable as the teleoperation happens not at a very low level but I would
like to reach sub-second latencies. The priority should be given to latency
and after to bandwidth as normally the streams travels on wifi LAN.

 

FFmpeg is launched with this command:

 

ffmpeg -hide_banner -f rawvideo -pixel_format gray -video_size 1280x960
-framerate 5 -i - -an http://localhost:8080/stream.ffm

 

and FFserver:

 

<Feed stream.ffm>

      File /tmp/stream.ffm

      FileMaxSize 200K

      ACL allow 192.168.0.0 192.168.255.255

</Feed>

 

<Stream XB3-240.h264>

      Feed stream.ffm

      Format rtp

      

      VideoCodec libx264

      PixelFormat yuv420p

      VideoFrameRate 4

      VideoSize 320x240

      

      AVOptionVideo crf 23

      AVOptionVideo profile baseline

 

      #VideoIntraOnly

      VideoGopSize 8

      

      NoAudio

      AVOptionAudio flags +global_header

</Stream>

 


 

A few questions:

-          I was unable to find out how to measure latency in every stage of
the stream, is there a clever method to do it different from quick and dirty
solutions as shown in the first link <http://fomori.org/blog/?p=1213> ? 

-          May you be able to suggest how to reduce latency as much as
possible at OS and ffmpeg/ffserver and network level.

-          I would like to stick to H264 but other option may be evaluated
in case they may bring much better results.

 

Thanks in advance for any help,

 

Best regards,

Federico Salvioli.