[FFmpeg-devel] [PATCH] libavutil/video_enc_params: add block type

Mark Thompson sw at jkqxz.net
Thu Jul 16 03:09:43 EEST 2020


On 16/07/2020 00:35, Lynne wrote:
> Jul 16, 2020, 00:06 by sw at jkqxz.net:
> 
>> On 15/07/2020 18:43, Yongle Lin wrote:
>>
>>> add block type field to AVVideoBlockParams so we could either export or visualize it later.
>>> ---
>>>   libavutil/video_enc_params.h | 19 +++++++++++++++++++
>>>   1 file changed, 19 insertions(+)
>>>
>>> diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h
>>> index 43fa443154..8bf5f240c9 100644
>>> --- a/libavutil/video_enc_params.h
>>> +++ b/libavutil/video_enc_params.h
>>> @@ -57,6 +57,11 @@ enum AVVideoEncParamsType {
>>>   AV_VIDEO_ENC_PARAMS_H264,
>>>   };
>>>   +enum AVVideoBlockFlags {
>>> +    AV_VIDEO_ENC_BLOCK_INTRA = 1ULL <<  0,  /* Indicates block uses intra prediction */
>>>
>>
>> The ULL is only confusing matters here - standard-conforming enum values have type int.  Some compilers allow it to be a larger integer type, but I don't think we can rely on that extension.
>>
> 
> We already rely on this behavior internally in hwcontext_vulkan. Are you sure there's a compiler
> we support that doesn't allow this? If so we ought to fix that.
You don't actually need it in hwcontext_vulkan, since it's internal-only (no ABI issues) and everything would work on a conforming compiler which just converts the values to int.

IMO it's not a useful enough feature to add to the special list of non-C99 things allowed.  Here it seems highly unlikely that we will get anywhere near 32 separate flags.

>>> +    AV_VIDEO_ENC_BLOCK_SKIP = 1ULL <<  1,   /* Indicates block is not coded (skipped) */
>>>
>>
>> Can you define more precisely what you mean by "not coded"?  Is that just that it has no residuals, or does it also indicate no motion vector, or no coded motion vector relative to prediction?
>>
> 
> In AV1 and VP9, which are AFAIK the only codecs to have this type of block skipping, are only able to
> code deblocking level adjustments for skipped blocks. No residual is sent, no motion vector
> either (hence the motion vector used is the likeliest candidate from the list of motion vectors).
> If a block isn't skipped, a motion vector may or may not be coded, depending on what's in
> the list of candidates for a block, so it would be inappropriate to apply the skip flag for anything
> but AV1/VP9-style skipped blocks.
> VP8 also supports all of that but no one cares about VP8 so its irrelevant.

There are block types P_Skip and B_Skip in H.264, which code nothing at all in the bitstream after the skip indication.

P_Skip uses the predicted motion vector against index 0 of RefPicList0.

B_Skip has a complex (but precisely-defined) procedure to try to choose a reference picture from each of the two RefPicLists and motion vectors in them, and will combine them as normal if it has both.

The first sounds exactly like what you're saying is in VP9/AV1.  The second is more complex, but should probably still count?  Dunno.

- Mark


More information about the ffmpeg-devel mailing list