[Libav-user] RTP/RTSP support in FFMPEG

[Luke Clemens]

I did a diff with an old version of rtpdec.c and a new version (the
old on has the timeout problem but it doesn't get mpeg4 errors). They
are very different - it's almost a complete rewrite. The code is
difficult to follow (well for me anyway, coming from an object
orientated C++ point of view). Also, I'm not an RTP expert so I need
to give myself a crash course with RFC3550 documentation and
wikipedia.

I tried setting the queue size (which was defaulting to zero). That
can be done in ffplay using something like -max_delay 500000. After
doing that I can now see that lots of packets are being dropped. My
next thought was that maybe the decoding was too slow so the socket
wasn't able receive fast enough. So I commented out the decoding
portion. Now I have a loop that's basically reading as fast as
possible and doing nothing with the frame.

while (1) {
     av_read_frame(ctx, pkt);
}

I set the log level to debug using av_set_log_level(AV_LOG_DEBUG).
Even without decoding there one or two bright yellow warnings every
second or so that look like:

[mpeg4 @ 0037b6a0] RTP: missed 3 packets
[pcm_mulaw @ 0037d980] RTP: missed 5 packets
[mpeg4 @ 0037b6a0] RTP: missed 1 packets
[mpeg4 @ 0037b6a0] RTP: missed 1 packets
[mpeg4 @ 0037b6a0] RTP: missed 2 packets
[mpeg4 @ 0037b6a0] RTP: missed 1 packets
[mpeg4 @ 0037b6a0] RTP: missed 3 packets
[mpeg4 @ 0037b6a0] RTP: missed 1 packets
[pcm_mulaw @ 0037d980] RTP: missed 3 packets
[mpeg4 @ 0037b6a0] RTP: missed 1 packets
[mpeg4 @ 0037b6a0] RTP: missed 1 packets
[mpeg4 @ 0037b6a0] RTP: missed 4 packets

I'm almost positive that the "drops" are not due to network or
hardware. The AXIS mpeg4 camera I'm testing with is plugged into the
same gigabit switch as my PC and I've sent WAY higher bandwidth RTP
streams through without any problems. Also, the older version of
libavformat didn't have this issue, nor does live555 which VLC uses.

Those warnings come from the rtp_parse_queued_packet() function in
rtpdec.c. It throws the warning when the following packet's sequence
number isn't the current sequence number plus one. Unless I'm missing
something, the rtp parsing code doesn't appear to account for packets
coming in out of order. Perhaps the AXIS and Panasonic cameras have a
problem sending packets in order? But if that were the case, I would
think the older versions of libavformat wouldn't have worked.  Wish I
knew the guy who wrote this code because he would understand it much
better...

The rtp_parse_queued_packet() function gets called by
ff_rtsp_fetch_packet() in rtsp.c, which is called by
rtsp_read_packet() in rtspdec.c.

The socket reading code occurs in ffurl_read(), which calls
retry_transfer_wrapper(), which calls transfer_func(), which calls a
callback in the URLContext structure, which performs udp_read(), which
calls recv() on a socket.

At this point, I don't know if the problem is in the rtsp decoder, or
if it's an issue with the actual UDP socket reading. If other UDP
transport protocols work, then that probably means it's not the UDP
reading at fault.

I also have a headache... :-)

This would be easier if I could get ffmpeg working with a visual
debugger. I tried it with Qt Creator in Ubuntu but ran into problems.
Does anyone know where I can find a tutorial or something on how to
visually debug libavformat or ffmpeg? I don't care if it's qt creator,
eclipse or visual studio - as long as I can step into ffmpeg
functions.

--luke