[Libav-user] trouble syncing with x264 and RTP

[Jacob Peddicord]

Hi,

I've been working on some streaming software that takes live feeds
from various kinds of cameras and streams over the network using
H.264. To accomplish this, I'm using the x264 encoder directly (with
the "zerolatency" preset) and feeding NALs as they are available to
libavformat to pack into RTP (ultimately RTSP). Ideally, this
application should be as real-time as possible. For the most part,
this has been working well.

Unfortunately, however, there is some sort of synchronization issue:
any video playback on clients seems to show a few smooth frames,
followed by a short pause, then more frames; repeat. Additionally,
there appears to be approximately a 4-second delay. This happens with
every video player I've tried: Totem, VLC, and basic gstreamer pipes.

I've boiled it all down to a somewhat small test case, which is
attached. This test shows black lines on a white background that
should move smoothly to the left. It has been written for ffmpeg 0.6.5
(sorry) but appears to work the same on 0.8.3 on Ubuntu. I've taken
some shortcuts in error handling to make this example as short as
possible while still showing the problem, so please excuse some of the
nasty code. I should also note that while an SDP is not used here, I
have tried using that already with similar results. The test can be
compiled with:

gcc -g -std=gnu99 streamtest.c -lswscale -lavformat -lx264 -lm
-lpthread -o streamtest

It can be played with gtreamer directly:

gst-launch udpsrc port=49990 !
application/x-rtp,payload=96,clock-rate=90000 ! rtph264depay !
decodebin ! xvimagesink

You should immediately notice the stuttering. One common "fix" I've
seen all over the Internet is to add sync=false to the pipeline:

gst-launch udpsrc port=49990 !
application/x-rtp,payload=96,clock-rate=90000 ! rtph264depay !
decodebin ! xvimagesink sync=false

This causes playback to be smooth (and near-realtime), but is a
non-solution and only works with gstreamer. I'd like to fix the
problem at the source. I've been able to stream with near-identical
parameters using raw ffmpeg and haven't had any issues:

ffmpeg -re -i sample.mp4 -vcodec libx264 -vpre ultrafast -vpre
baseline -b 400000 -an -f rtp rtp://127.0.0.1:49990 -an

So clearly I'm doing something wrong. But what is it? Any assistance
would be greatly appreciated.

-- 
Jacob Peddicord
http://jacob.peddicord.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: streamtest.c
Type: text/x-csrc
Size: 4280 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20120724/67820c66/attachment.bin>