[FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: add hw_configs

[Zhao Zhili]

> On Oct 12, 2023, at 23:09, Zhao Zhili <quinkblack at foxmail.com> wrote:
> 
> 
>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Andreas Rheinhardt
>> Sent: 2023年10月12日 22:56
>> To: ffmpeg-devel at ffmpeg.org
>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: add hw_configs
>> 
>> Zhao Zhili:
>>> 
>>>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Andreas Rheinhardt
>>>> Sent: 2023年10月12日 22:01
>>>> To: ffmpeg-devel at ffmpeg.org
>>>> Subject: Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: add hw_configs
>>>> 
>>>> Zhao Zhili:
>>>>> From: Zhao Zhili <zhilizhao at tencent.com>
>>>>> 
>>>>> Will be used in the following patches.
>>>> 
>>>> It doesn't seem to be used for 2/2, so it should be applied with the
>>>> patches that actually need it.
>>> 
>>> It's used in an nonobvious way. With hw_config we can get avctx->hw_frames_ctx,
>>> and with avctx->hw_frames_ctx we get sw_pix_fmt. Otherwise sw_pix_fmt is none.
>>> I need sw_pix_fmt before get the first frame.
>>> 
>> 
>> So setting these hw_config magically sets AVCodecContext.sw_pix_fmt? How
>> does this work? Does this work magically with callers other than the
>> ffmpeg cli?
> 
> It's done by encode.c encode_preinit_video() to copy hw_frames_ctx sw_format to avctx->sw_pix_fmt.
> If user handles the avcodec_get_hw_config() and set hw_frames_ctx, it will work. It doesn't work if
> hw_frames_ctx is null, and hw_config is a capability and hint, not a hard requirement for all encoders.
> Although we can enforce a valid hw_frames_ctx during videotoolbox encoder init, then it can break
> use's code for no good reasons.

Will apply the patchset tomorrow with some description on why the first patch is needed.

> 
>> 
>>>> 
>>>>> ---
>>>>> libavcodec/videotoolboxenc.c | 9 +++++++++
>>>>> 1 file changed, 9 insertions(+)
>>>>> 
>>>>> diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
>>>>> index 1d1595329a..b685bf4cc4 100644
>>>>> --- a/libavcodec/videotoolboxenc.c
>>>>> +++ b/libavcodec/videotoolboxenc.c
>>>>> @@ -37,6 +37,7 @@
>>>>> #include "encode.h"
>>>>> #include "h264.h"
>>>>> #include "h264_sei.h"
>>>>> +#include "hwconfig.h"
>>>>> #include <dlfcn.h>
>>>>> 
>>>>> #if !HAVE_KCMVIDEOCODECTYPE_HEVC
>>>>> @@ -2830,6 +2831,11 @@ static const enum AVPixelFormat prores_pix_fmts[] = {
>>>>>         "Sets the maximum number of reference frames. This only has an effect when the value is less than the maximum
>> allowed
>>>> by the profile/level.", \
>>>>>         OFFSET(max_ref_frames), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, VE },
>>>>> 
>>>>> +static const AVCodecHWConfigInternal *const vt_encode_hw_configs[] = {
>>>>> +        HW_CONFIG_ENCODER_FRAMES(VIDEOTOOLBOX, VIDEOTOOLBOX),
>>>>> +        NULL,
>>>>> +};
>>>>> +
>>>>> #define OFFSET(x) offsetof(VTEncContext, x)
>>>>> static const AVOption h264_options[] = {
>>>>>     { "profile", "Profile", OFFSET(profile), AV_OPT_TYPE_INT, { .i64 = AV_PROFILE_UNKNOWN }, AV_PROFILE_UNKNOWN,
>> INT_MAX,
>>>> VE, "profile" },
>>>>> @@ -2886,6 +2892,7 @@ const FFCodec ff_h264_videotoolbox_encoder = {
>>>>>     .close            = vtenc_close,
>>>>>     .p.priv_class     = &h264_videotoolbox_class,
>>>>>     .caps_internal    = FF_CODEC_CAP_INIT_CLEANUP,
>>>>> +    .hw_configs       = vt_encode_hw_configs,
>>>>> };
>>>>> 
>>>>> static const AVOption hevc_options[] = {
>>>>> @@ -2923,6 +2930,7 @@ const FFCodec ff_hevc_videotoolbox_encoder = {
>>>>>     .p.priv_class     = &hevc_videotoolbox_class,
>>>>>     .caps_internal    = FF_CODEC_CAP_INIT_CLEANUP,
>>>>>     .p.wrapper_name   = "videotoolbox",
>>>>> +    .hw_configs       = vt_encode_hw_configs,
>>>>> };
>>>>> 
>>>>> static const AVOption prores_options[] = {
>>>>> @@ -2961,4 +2969,5 @@ const FFCodec ff_prores_videotoolbox_encoder = {
>>>>>     .p.priv_class     = &prores_videotoolbox_class,
>>>>>     .caps_internal    = FF_CODEC_CAP_INIT_CLEANUP,
>>>>>     .p.wrapper_name   = "videotoolbox",
>>>>> +    .hw_configs       = vt_encode_hw_configs,
>>>>> };
>>>> 
>> 
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".