[FFmpeg-devel] [PATCH 3/3] avcodec/mediacodecenc: workaround the alignment requirement for H.265

[Zhao Zhili]

> On May 23, 2024, at 15:08, Zhao Zhili <quinkblack at foxmail.com> wrote:
> 
> From: Zhao Zhili <zhilizhao at tencent.com>
> 
> Signed-off-by: Zhao Zhili <zhilizhao at tencent.com>
> ---
> libavcodec/mediacodecenc.c | 11 ++++++++---
> 1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c
> index bfff149039..a42270551b 100644
> --- a/libavcodec/mediacodecenc.c
> +++ b/libavcodec/mediacodecenc.c
> @@ -154,8 +154,12 @@ static int mediacodec_init_bsf(AVCodecContext *avctx)
>             ret = snprintf(str, sizeof(str), "h264_metadata=crop_right=%d:crop_bottom=%d",
>                            crop_right, crop_bottom);
>         else if (avctx->codec_id == AV_CODEC_ID_HEVC)
> -            ret = snprintf(str, sizeof(str), "hevc_metadata=crop_right=%d:crop_bottom=%d",
> -                           crop_right, crop_bottom);
> +            /* Encoder can use CTU size larger than 16x16, so the real crop
> +             * margin can be larger than crop_right/crop_bottom. Let bsf figure
> +             * out the real crop margin.
> +             */
> +            ret = snprintf(str, sizeof(str), "hevc_metadata=width=%d:height=%d",
> +                           avctx->width, avctx->height);
>         if (ret >= sizeof(str))
>             return AVERROR_BUFFER_TOO_SMALL;
>     }
> @@ -235,7 +239,8 @@ static av_cold int mediacodec_init(AVCodecContext *avctx)
>     // Workaround the alignment requirement of mediacodec. We can't do it
>     // silently for AV_PIX_FMT_MEDIACODEC.
>     if (avctx->pix_fmt != AV_PIX_FMT_MEDIACODEC &&
> -        avctx->codec_id == AV_CODEC_ID_H264) {
> +        (avctx->codec_id == AV_CODEC_ID_H264 ||
> +         avctx->codec_id == AV_CODEC_ID_HEVC)) {
>         s->width = FFALIGN(avctx->width, 16);
>         s->height = FFALIGN(avctx->height, 16);
>     } else {
> -- 
> 2.42.0
> 

For anyone who is interested on why the workaround is important, here is an example.

This mediacodec xml comes from a pretty new mobile made by Vivo, with CPU from MTK.
Both H.264 and H.265 encoder require alignment of 16x16. MediaCodec will fail during
configure with resolution like 1080x1920. When use surface as input, it will output a video
stream with resolution 1072x1920. It’s weird, but it’s not rare.

  <MediaCodec name="c2.mtk.avc.encoder" type="video/avc" >
            <Alias name="OMX.MTK.VIDEO.ENCODER.AVC" />
            <Limit name="size" min="160x128" max="3840x2176" />
            <Limit name="alignment" value="16x16" />
            <Limit name="block-size" value="16x16" />
            <Limit name="concurrent-instances" max="10" />
            <Limit name="performance-point-3840x2160" value="30" />
            <Limit name="performance-point-1280x719" value="140" />
            <Limit name="performance-point-1280x720" value="120" />
        </MediaCodec>

  <MediaCodec name="c2.mtk.hevc.encoder" type="video/hevc" >
            <Alias name="OMX.MTK.VIDEO.ENCODER.HEVC" />
            <Limit name="size" min="160x128" max="7680x4320" />
            <Limit name="alignment" value="16x16" />
            <Limit name="block-size" value="16x16" />
            <Limit name="quality" range="0-100" default="50" />
            <Limit name="concurrent-instances" max="10" />
            <Feature name="bitrate-modes" value="VBR,CBR,CQ" />
            <Limit name="performance-point-3840x2160" value="30" />
            <Limit name="performance-point-1280x719" value="140" />
            <Limit name="performance-point-1280x720" value="120" />
        </MediaCodec>