[Libav-user] Force low-delay handling? question + feature suggestion

[Camera Man]

On 03/11/2012 11:57 AM, Carl Eugen Hoyos wrote:
> You would have to discard all B-frames (because they require 
> reordering) not only the non-ref ones, i.e. corruption would be possible.

That's cool! I didn't know that B frames can be reference frames (The 
last spec I read top-to-bottom was mpeg2, where B frames were never 
references). Although it foils my feature suggestion...

> Did you write such a patch?

I forced has_b_frames to 0 and low_delay to 1 with a patch in the 
ugliest possible way (all over the places that these may be changed, 
disregarding the discard setting), and it worked for the streams 
produced by my camera (which claims max redorder frames = 4, but I've 
never actually seen anything that was out of order), but I'm afraid to 
use it in practice -- the camera says "num_reorder = 4", and although I 
haven't been able to produce anything out-of-order from it, there might 
be circumstances it does.

I have no idea how the decoder would behave in this case, and no 
reasonable way to test.

Another possible solution for my use case, which might be useful for 
others, is to somehow pull the latest-pts picture from the insides of 
the h264; something like:

    avcodec_decode_video2(context, frame1, packet, &frame1_finished);
    avcodec_h264_internal_fetch_most_recent(context, frame2, 
&frame2_finished); /* this is what I would need to write */

    switch (2*(frame2_finished!=0)+1*(frame1_finished!=0)) {
       case 0: /* none at all */ break;
       case 1: /* frame1, no frame2 */ use(frame1);
       case 2: /* frame2, no frame1 */ use(frame2);
       case 3: /* both; pick latest */ use(frame1->pts >= frame2->pts ? 
frame1 : frame2);
    }

Where avcodec_h264_internal_fetch_most_recent() would get a copy of the 
highest pts frame in the delay buffer, without changing or discarding 
it. If this is possible (and I manage to write this ... I'm not sure I 
understand all of h264 internals), then it would introduce a "lowest 
delay" from bitstream to display - at the cost of not showing any 
out-of-order frames (having an effectively lower frame rate - but much 
better real-time latency).

Can you think of a reason off-hand this wouldn't work?

> Please note that your mails are nearly unreadable:
> http://ffmpeg.org/pipermail/libav-user/2012-March/001495.html
>
> Consider setting your mailer to "plain text".

Thanks - even though my mailer was set to "Send both HTML and plain 
text", for some reason to ffmpeg.org it insists on sending HTML only. 
I've set it to plain-text only, in the hope that it would actually work 
this time. Apologies for the previously unreadable emails.