[Libav-user] Criteria for h.264 key-frame detection

[Robert Krüger]

On Sun, Oct 20, 2013 at 2:44 PM, Carl Eugen Hoyos <cehoyos at ag.or.at> wrote:
> Robert Krüger <krueger at ...> writes:
>
>> So are you saying that AVPacket.key_frame should be
>> set to 1 for recovery points in H.264?
>
> No, on the contrary, I don't think this would "work"
> (depending on the definition of work) in the sense
> that the frame marked as key_frame cannot be
> independently decoded if all recovery points were
> marked as key_frames.
> What I was trying to say is that the combination
> of libavformat and libavcodec supports recovery
> points, ie the user can choose if he wants to seek
> to a recovery point or to some frame (no matter if
> it can be independently decoded).
> I don't think I understand the other use case of
> recovery frames that you apparently have in mind.

Suppose I just wanted to find out which valid recovery points are for
a transport stream file by reading the output of the demuxer (e.g. to
display this in an analysis report) or any other means if you have
another suggestion.

Looking at the API documentation, h264 code and debugging a bit this
is what I have found so far. Maybe it helps someone else.

The h.264 parser sets key_frame = 1 (AVCodecParserContext) for packets
following a recovery point SEI (even if recovery_frame_cnt is > 1 and
regardless of the value of the exact_match_flag, which is not parsed)
and that triggers the AVPacket's AV_PKT_FLAG_KEY being set. After
checking docs for a second time, I would say this is consistent with
the documentation of AVPacket.convergence_duration.

This is the behavior I can see with the files described in ticket
#3063, but that is not the issue here. For those files AFAICS, the
demuxer side works as documented.

In case of the files I am talking about here, there are no recovery
point SEIs after the first frame so libavformat/avcodec is probably
doing everything correctly, too. Since I think I know that these
streams in fact do have random access points, I will probably have to
parse the h264 metadata myself and apply some heuristics in my
application but filing a ticket does not seem to make sense as it does
not appear to be a defect and I don't think anyone would have an
interest to work on such heuristics or further investigation, how
random access points can be found in such a stream (if they exist,
which I have yet to prove).

>
> (The disclaimer still applies but I wonder if you
> know that a valid H264 stream does not necessarily
> contain keyframes but still allows to seek without
> corruption afaiu.)

I do.