[FFmpeg-user] Difference between i-frames and key frames

[Mark Filipak]

On 10/30/23 14:16, Nicolas Gaullier wrote:
>>> On 07-10-2023 17:20, Ulf Zibis wrote:
>>>> Hi,
>>>>
>>>> what is the difference between i-frames and key frames?
>>>>
>>>> And how many other frames are between i-frames or key frames in mp4
>>>> files, created with ffmpeg and with default settings?
>>>>
>>>> -Ulf
>>> In mp4 containers I-frames are key frames.>
>>
>> That is not correct. Via ffprobe, I have seen I-frames that were not marked 'key_frame=1'.
>
> Firstly, the definition of what a key frame is may depend on contexts and on peoples,
> it is often a source of headaches.
> If you take a look at
> https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/h264_parser.c
> You can see that properly signaled recovery points are tagged as keyframes,
> which means you don't need an I-frame to make a keyframe (some low latency encoders make use of this).
> And you can also see that kind of things:
> // heuristic to detect non marked keyframes
> if (p->ps.sps->ref_frame_count <= 1 && p->ps.pps->ref_count[0] <= 1 && s->pict_type == AV_PICTURE_TYPE_I)
> s->key_frame = 1;
> as indeed, an I frame is not always a keyframe (long term references may be required later for further processing).
>
> And, at the end, sorry for not answering your question ;)
>
> Nicolas

Thank you, Nicolas. Note that I code, but not 'C'.

Regarding 'https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/h264_parser.c'
line#
/
273 s->pict_type=AV_PICTURE_TYPE_I;
274 s->key_frame=0;
338 switch(nal.type)
{
349 case H264_NAL_IDR_SLICE:
350 s->key_frame=1;
357 case H264_NAL_SLICE:
361 if(p->sei.recovery_point.recovery_frame_cnt>=0)s->key_frame=1;
382 if
( p->ps.sps->ref_frame_count<=1
&&p->ps.pps->ref_count[0]<=1
&&s->pict_type==AV_PICTURE_TYPE_I
)s->key_frame=1;
571 }

What is 's'?
What is 'p'?
What does '->' mean?/do? [note 1]
What is 'ps'?
What is 'ps.sps'?
What is H264_NAL_IDR_SLICE? [note 2]
What is H264_NAL_SLICE?
What is recovery_point?

I think I know (guess) what everything else is/means.

[note 1] Yes, I think that's a pointer. Please don't waste your time
explaining pointers.
[note 2] I've never seen an explanation what 'NAL' means. I think
what's meant is 'network adaption layer', though I don't know what it
has to do with networking, so I'm probably wrong.

I must say that I don't care for the code. For example, 'pict_type' is
initialized to 'AV_PICTURE_TYPE_I' and there's nothing between that,
and the 'switch', and the 'case' that changes it, yet the case seems
to test 'pict_type==AV_PICTURE_TYPE_I'. Without knowing 'C', I
nonetheless have taught 'C' codesmiths how to build state machines
that have 100% coverage and are 100% testable. In my experience,
almost all bugs come from 'accidental' state machines using
'unrecognized' state variables implemented by threaded ('spaghetti')
code.

--Mark.