[FFmpeg-devel] rtp streaming x264+audio issues (and some ideas to fix them)

[Timo Teräs]

Luca Abeni wrote:
> Martin Storsj? wrote:
> [...]
>>> +    s->first_rtcp_ntp_time = ntp_time();
>>>
>>>     max_packet_size = url_fget_max_packet_size(s1->pb);
>>>     if (max_packet_size <= 12)
>>>
>>> What would be the proper way to fix the synchronization issue?
>>> So that the RTP packets sent out are approximately (<1s difference)
>>> from the original stream.
>>
>> I've also been looking at this same issue, and this is IMO the first 
>> step towards getting it right. This way, the sync error between the 
>> two streams is reduced to the duration between the calls to 
>> rtp_write_header
> 
> I had a look at the patch and at the current code. After thinking a little
> bit about it, I think the patch is good and should be committed. After all,
> requiring that rtp_write_header() calls for the various streams are happen
> in a short time seems reasonable. And should (IMHO) work in 99% of the
> cases.

Seems like I got a nice thread started :) ... But yes, it was intended
to show what is broke. And be just a quick fix for the common case; well
at least make it a whole lot better than what it was.

Sending the rtp streams 1-2 secs apart can be problematic for some
clients, but that's another issue.

> So, I'll commit this patch (well, not really this patch... But something
> similar: if first_rtcp_ntp_time is initialised in rtp_write_header(),

I guess it should look something like:

--- libavformat/rtpenc.c	(revision 21686)
+++ libavformat/rtpenc.c	(working copy)
@@ -91,7 +91,7 @@
     s->cur_timestamp = 0;
     s->ssrc = 0; /* FIXME: was random(), what should this be? */
     s->first_packet = 1;
-    s->first_rtcp_ntp_time = AV_NOPTS_VALUE;
+    s->first_rtcp_ntp_time = ntp_time();
 
     max_packet_size = url_fget_max_packet_size(s1->pb);
     if (max_packet_size <= 12)
@@ -171,7 +171,6 @@
 
     dprintf(s1, "RTCP: %02x %"PRIx64" %x\n", s->payload_type, ntp_time, s->timestamp);
 
-    if (s->first_rtcp_ntp_time == AV_NOPTS_VALUE) s->first_rtcp_ntp_time = ntp_time;
     s->last_rtcp_ntp_time = ntp_time;
     rtp_ts = av_rescale_q(ntp_time - s->first_rtcp_ntp_time, (AVRational){1, 1000000},
                           s1->streams[0]->time_base) + s->base_timestamp;

- Timo