[FFmpeg-devel] [PATCH] lavc/qsvenc: add Tiles encode support for HEVC
linjie fu
linjie.justin.fu at gmail.com
Sat Dec 7 08:28:45 EET 2019
Hi,
On Fri, Nov 29, 2019 at 2:42 PM Fu, Linjie <linjie.fu at intel.com> wrote:
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > Zhong Li
> > Sent: Friday, November 29, 2019 13:13
> > To: FFmpeg development discussions and patches <ffmpeg-
> > devel at ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc: add Tiles encode support
> > for HEVC
> >
> > Linjie Fu <linjie.fu at intel.com> 于2019年11月26日周二 下午12:04写道：
> > >
> > > Add -tile_rows and -tile_cols option to specify the number of tile rows
> > > and columns for ICL+ (gen 11) platform.
> > >
> > > A tile must wholly contain all the slices within it. Slices cannot
> cross
> > > tile boundaries. So the slice number would be implicitly resized to the
> > > max(nSlice, nTile).
> >
> > I would like to the check in the code too, instead of just commit
> message.
>
> Thanks, code link in MSDK should have been attached too:
>
> https://github.com/Intel-Media-SDK/MediaSDK/blob/5a84af5fa61823a84f32e4b26d864730a927f48c/_studio/mfx_lib/encode_hw/h265/src/mfx_h265_encode_hw_par.cpp#L503
>
> This is merged in：
>
> https://github.com/Intel-Media-SDK/MediaSDK/commit/f8ba11a5b7d61422b117bec2b5b88e4a22bbd294
>
> >
> > > Example:
> > > ffmpeg -v verbose -hwaccel qsv -init_hw_device qsv=hw
> > > -filter_hw_device hw -f rawvideo -s:v 1920x1080 -i ./input.nv12 -vf
> > > format=nv12,hwupload=extra_hw_frames=64 -c:v hevc_qsv -tile_rows 2
> > > -tile_cols 2 -slices 4 -y output.h265
> > >
> > > Also dump the actual quantity of encoded tiled rows and columns in run
> > > time.
> > >
> > > Fix the enhancement #8400.
> > >
> > > Signed-off-by: Linjie Fu <linjie.fu at intel.com>
> > > ---
> > > libavcodec/qsvenc.c | 32 +++++++++++++++++++++++++++++++-
> > > libavcodec/qsvenc.h | 7 +++++++
> > > libavcodec/qsvenc_hevc.c | 3 +++
> > > 3 files changed, 41 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
> > > index e176d57..64814fc 100644
> > > --- a/libavcodec/qsvenc.c
> > > +++ b/libavcodec/qsvenc.c
> > > @@ -139,6 +139,9 @@ static void dump_video_param(AVCodecContext
> > *avctx, QSVEncContext *q,
> > > #if QSV_HAVE_CO3
> > > mfxExtCodingOption3 *co3 = (mfxExtCodingOption3*)coding_opts[2];
> > > #endif
> > > +#if QSV_HAVE_EXT_HEVC_TILES
> > > + mfxExtHEVCTiles *exthevctiles = (mfxExtHEVCTiles *)coding_opts[3 +
> > QSV_HAVE_CO_VPS];
> >
> > Should change QSV_HAVE_CO_VPS to QSV_HAVE_EXT_HEVC_TILES?
>
> There is possibility(MSDK version between 1.13 and 1.17) that MSDK has
> MFX_EXTBUFF_HEVC_TILES buffer but has no MFX_EXT_BUFF_CODING_OPTION_VPS
> buffer.
>
> ext_buffers[]:
> BUF_CO | BUF_CO2 | BUF_CO3 | BUF_VPS | BUF_TILE |...
>
> Hence I'd like to check QSV_HAVE_CO_VPS to get the exact address of
> BUF_TILE
> to dump the truly set values.
>
> >
> > > +#endif
> > >
> > > av_log(avctx, AV_LOG_VERBOSE, "profile: %s; level: %"PRIu16"\n",
> > > print_profile(info->CodecProfile), info->CodecLevel);
> > > @@ -204,6 +207,12 @@ static void dump_video_param(AVCodecContext
> > *avctx, QSVEncContext *q,
> > > av_log(avctx, AV_LOG_VERBOSE, "RateDistortionOpt: %s\n",
> > > print_threestate(co->RateDistortionOpt));
> > >
> > > +#if QSV_HAVE_EXT_HEVC_TILES
> > > + if (avctx->codec_id == AV_CODEC_ID_HEVC)
> > > + av_log(avctx, AV_LOG_VERBOSE, "NumTileColumns: %"PRIu16";
> > NumTileRows: %"PRIu16"\n",
> > > + exthevctiles->NumTileColumns,
> exthevctiles->NumTileRows);
> > > +#endif
> > > +
> > > #if QSV_HAVE_CO2
> > > av_log(avctx, AV_LOG_VERBOSE,
> > > "RecoveryPointSEI: %s IntRefType: %"PRIu16";
> > IntRefCycleSize: %"PRIu16"; IntRefQPDelta: %"PRId16"\n",
> > > @@ -771,6 +780,16 @@ FF_ENABLE_DEPRECATION_WARNINGS
> > > }
> > > #endif
> > >
> > > +#if QSV_HAVE_EXT_HEVC_TILES
> > > + if (avctx->codec_id == AV_CODEC_ID_HEVC) {
> > > + q->exthevctiles.Header.BufferId = MFX_EXTBUFF_HEVC_TILES;
> > > + q->exthevctiles.Header.BufferSz = sizeof(q->exthevctiles);
> > > + q->exthevctiles.NumTileColumns = q->tile_cols;
> > > + q->exthevctiles.NumTileRows = q->tile_rows;
> > > + q->extparam_internal[q->nb_extparam_internal++] =
> (mfxExtBuffer
> > *)&q->exthevctiles;
> > > + }
> > > +#endif
> > > +
> > > if (!check_enc_param(avctx,q)) {
> > > av_log(avctx, AV_LOG_ERROR,
> > > "some encoding parameters are not supported by the QSV
> "
> > > @@ -887,7 +906,14 @@ static int
> > qsv_retrieve_enc_params(AVCodecContext *avctx, QSVEncContext *q)
> > > };
> > > #endif
> > >
> > > - mfxExtBuffer *ext_buffers[2 + QSV_HAVE_CO2 + QSV_HAVE_CO3 +
> > QSV_HAVE_CO_VPS];
> > > +#if QSV_HAVE_EXT_HEVC_TILES
> > > + mfxExtHEVCTiles hevc_tile_buf = {
> > > + .Header.BufferId = MFX_EXTBUFF_HEVC_TILES,
> > > + .Header.BufferSz = sizeof(hevc_tile_buf),
> > > + };
> > > +#endif
> > > +
> > > + mfxExtBuffer *ext_buffers[2 + QSV_HAVE_CO2 + QSV_HAVE_CO3 +
> > QSV_HAVE_CO_VPS + QSV_HAVE_EXT_HEVC_TILES];
> > >
> > > int need_pps = avctx->codec_id != AV_CODEC_ID_MPEG2VIDEO;
> > > int ret, ext_buf_num = 0, extradata_offset = 0;
> > > @@ -905,6 +931,10 @@ static int
> > qsv_retrieve_enc_params(AVCodecContext *avctx, QSVEncContext *q)
> > > if (q->hevc_vps)
> > > ext_buffers[ext_buf_num++] = (mfxExtBuffer*)&extradata_vps;
> > > #endif
> > > +#if QSV_HAVE_EXT_HEVC_TILES
> > > + if (avctx->codec_id == AV_CODEC_ID_HEVC)
> > > + ext_buffers[ext_buf_num++] = (mfxExtBuffer*)&hevc_tile_buf;
> > > +#endif
> > >
> > > q->param.ExtParam = ext_buffers;
> > > q->param.NumExtParam = ext_buf_num;
> > > diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h
> > > index ee35582..6609171 100644
> > > --- a/libavcodec/qsvenc.h
> > > +++ b/libavcodec/qsvenc.h
> > > @@ -38,6 +38,7 @@
> > > #define QSV_HAVE_CO3 QSV_VERSION_ATLEAST(1, 11)
> > > #define QSV_HAVE_CO_VPS QSV_VERSION_ATLEAST(1, 17)
> > >
> > > +#define QSV_HAVE_EXT_HEVC_TILES QSV_VERSION_ATLEAST(1, 13)
> > > #define QSV_HAVE_EXT_VP9_PARAM QSV_VERSION_ATLEAST(1, 26)
> > >
> > > #define QSV_HAVE_TRELLIS QSV_VERSION_ATLEAST(1, 8)
> > > @@ -124,6 +125,9 @@ typedef struct QSVEncContext {
> > > mfxExtMultiFrameParam extmfp;
> > > mfxExtMultiFrameControl extmfc;
> > > #endif
> > > +#if QSV_HAVE_EXT_HEVC_TILES
> > > + mfxExtHEVCTiles exthevctiles;
> > > +#endif
> > > #if QSV_HAVE_EXT_VP9_PARAM
> > > mfxExtVP9Param extvp9param;
> > > #endif
> > > @@ -161,6 +165,9 @@ typedef struct QSVEncContext {
> > > int max_frame_size;
> > > int max_slice_size;
> > >
> > > + int tile_cols;
> > > + int tile_rows;
> > > +
> > > int aud;
> > >
> > > int single_sei_nal_unit;
> > > diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c
> > > index da64b4c..27e2232 100644
> > > --- a/libavcodec/qsvenc_hevc.c
> > > +++ b/libavcodec/qsvenc_hevc.c
> > > @@ -243,6 +243,9 @@ static const AVOption options[] = {
> > >
> > > { "gpb", "1: GPB (generalized P/B frame); 0: regular P frame",
> > OFFSET(qsv.gpb), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, VE},
> > >
> > > + { "tile_cols", "Number of columns for tiled encoding",
> > OFFSET(qsv.tile_cols), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX,
> VE },
> >
> > should better to keep consist with other codecs such as vp9/av1 to set
> > the name as tile_columns.
> >
> > And I note the value is log2 in these codecs.
>
> Yes, I've noticed the name and values in log2 formats.
> BTW, MSDK allows Tile structures (such as 3x3 Tiles) instead of only 2^..
> Tiles,
> would it be better to use log2 format to set the rows/columns?
>
> > > + { "tile_rows", "Number of rows for tiled encoding",
> > OFFSET(qsv.tile_rows), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX,
> VE },
> > > +
> > > { NULL },
> > > };
> > >
> > > --
>
>
Ping,
More information about the ffmpeg-devel
mailing list